2024-03-23T12:58:04.452 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-23T12:58:04.452 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291 --name pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} --verbose -- /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291/orig.config.yaml 2024-03-23T12:58:04.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T12:58:04.597 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291 branch: wip-batrick-testing-20240323.003144-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} email: pdonnell@redhat.com first_in_suite: false job_id: '7618291' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240323.003144-squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 21b3bd672163092b4e542f772797fcba02d3d4de subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 21b3bd672163092b4e542f772797fcba02d3d4de kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240323.003144-squid sha1: d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4284 sha1: 21b3bd672163092b4e542f772797fcba02d3d4de sleep_before_teardown: 0 subset: 405/512 suite: fs suite_branch: wip-batrick-testing-20240323.003144-squid suite_path: /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPPS4kGbgZtyiA+PbFpN9jAvHO0clAoZR9dS4rFiGkjqcEPhEXfJ62lARiOgP+9sWl5LUv1ttet2I5YWUeaw2MI= smithi130.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIf1WrqU4CbY9z2YzSoXc8L4M0vncfJQIXiNx6UniD6JWDZsBmBtXNdL0EaJM1pHAbuvfFyzvGsdrPDTUR0bQrM= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKdQLOmBh37R1q5FxDRS0DC7QXWyZqMOUZX08w3fbQUA2sdfLy2ujTeb2MpJqnnoZy6dpZULJ8Xc697TLbOY5e8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-23_09:56:53 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-23T12:58:04.597 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa; will attempt to use it 2024-03-23T12:58:04.598 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks 2024-03-23T12:58:04.598 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-23T12:58:04.599 INFO:teuthology.task.internal:Checking packages... 2024-03-23T12:58:04.622 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '21b3bd672163092b4e542f772797fcba02d3d4de' 2024-03-23T12:58:04.623 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T12:58:04.623 INFO:teuthology.packaging:ref: None 2024-03-23T12:58:04.623 INFO:teuthology.packaging:tag: None 2024-03-23T12:58:04.623 INFO:teuthology.packaging:branch: wip-batrick-testing-20240323.003144-squid 2024-03-23T12:58:04.623 INFO:teuthology.packaging:sha1: 21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T12:58:04.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240323.003144-squid 2024-03-23T12:58:04.770 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1855.g21b3bd67 2024-03-23T12:58:04.771 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-23T12:58:04.780 INFO:teuthology.task.internal:no buildpackages task found 2024-03-23T12:58:04.780 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-23T12:58:04.795 INFO:teuthology.task.internal:Saving configuration 2024-03-23T12:58:04.813 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-23T12:58:04.820 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T12:58:04.843 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 12:52:45.252911', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPPS4kGbgZtyiA+PbFpN9jAvHO0clAoZR9dS4rFiGkjqcEPhEXfJ62lARiOgP+9sWl5LUv1ttet2I5YWUeaw2MI='} 2024-03-23T12:58:04.864 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 12:52:45.249833', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIf1WrqU4CbY9z2YzSoXc8L4M0vncfJQIXiNx6UniD6JWDZsBmBtXNdL0EaJM1pHAbuvfFyzvGsdrPDTUR0bQrM='} 2024-03-23T12:58:04.887 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 12:52:45.251610', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKdQLOmBh37R1q5FxDRS0DC7QXWyZqMOUZX08w3fbQUA2sdfLy2ujTeb2MpJqnnoZy6dpZULJ8Xc697TLbOY5e8='} 2024-03-23T12:58:04.887 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-23T12:58:04.894 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-23T12:58:04.894 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-23T12:58:04.894 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-23T12:58:04.894 INFO:teuthology.run_tasks:Running task console_log... 2024-03-23T12:58:04.998 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f20dbae2160>, signals=[15]) 2024-03-23T12:58:04.999 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-23T12:58:05.005 INFO:teuthology.task.internal:Opening connections... 2024-03-23T12:58:05.005 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-03-23T12:58:05.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T12:58:05.119 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi130.front.sepia.ceph.com 2024-03-23T12:58:05.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T12:58:05.208 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-03-23T12:58:05.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T12:58:05.294 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-23T12:58:05.300 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-03-23T12:58:05.318 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-03-23T12:58:05.319 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-03-23T12:58:05.376 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-03-23T12:58:05.376 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-03-23T12:58:05.376 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T12:58:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T12:58:05.378 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-03-23T12:58:05.416 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2024-03-23T12:58:05.434 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2024-03-23T12:58:05.434 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2024-03-23T12:58:05.490 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="CentOS Stream" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="9" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:ID="centos" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="rhel fedora" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="9" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el9" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:LOGO="fedora-logo-icon" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://centos.org/" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T12:58:05.491 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T12:58:05.492 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T12:58:05.492 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2024-03-23T12:58:05.516 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-03-23T12:58:05.534 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-03-23T12:58:05.535 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-03-23T12:58:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2024-03-23T12:58:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2024-03-23T12:58:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2024-03-23T12:58:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2024-03-23T12:58:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2024-03-23T12:58:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2024-03-23T12:58:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T12:58:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-03-23T12:58:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2024-03-23T12:58:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T12:58:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2024-03-23T12:58:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T12:58:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T12:58:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T12:58:05.591 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-03-23T12:58:05.616 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-23T12:58:05.627 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-23T12:58:05.632 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-23T12:58:05.633 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-03-23T12:58:05.635 DEBUG:teuthology.orchestra.run.smithi130:> test '!' -e /home/ubuntu/cephtest 2024-03-23T12:58:05.637 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-03-23T12:58:05.653 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-23T12:58:05.659 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-23T12:58:05.659 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-03-23T12:58:05.692 DEBUG:teuthology.orchestra.run.smithi130:> test -z $(ls -A /var/lib/ceph) 2024-03-23T12:58:05.695 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-03-23T12:58:05.797 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-23T12:58:05.875 INFO:teuthology.run_tasks:Running task kernel... 2024-03-23T12:58:05.890 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-23T12:58:05.890 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-23T12:58:05.890 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-23T12:58:05.890 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T12:58:05.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T12:58:05.908 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-03-23T12:58:05.966 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-361.el9.x86_64 2024-03-23T12:58:05.966 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-361.el9.x86_64 2024-03-23T12:58:05.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-03-23T12:58:08.094 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-23T12:58:09.527 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2024-03-23T12:58:14.468 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.0 MB 00:04 2024-03-23T12:58:16.502 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-23T12:58:21.353 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.1 MB 00:01 2024-03-23T12:58:22.837 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-03-23T12:58:24.882 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-23T12:58:30.038 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-03-23T12:58:32.162 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-23T12:58:32.162 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-23T12:58:32.256 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T12:58:32.272 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-23T12:58:32.273 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-23T12:58:32.273 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-03-23T12:58:32.273 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T12:58:32.273 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 76 M 2024-03-23T12:58:32.273 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 123 M 2024-03-23T12:58:32.273 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-23T12:58:32.898 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 126 kB/s | 44 kB 00:00 2024-03-23T12:58:46.763 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 1.0 MB/s | 15 MB 00:14 2024-03-23T12:58:58.920 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 1.1 MB/s | 27 MB 00:26 2024-03-23T12:59:10.919 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 893 kB/s | 33 MB 00:38 2024-03-23T12:59:10.919 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-23T12:59:10.919 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.0 MB/s | 76 MB 00:38 2024-03-23T12:59:11.610 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-23T12:59:12.035 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-23T12:59:12.035 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-23T12:59:13.095 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-23T12:59:13.095 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-23T12:59:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-23T12:59:16.326 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-23T12:59:16.413 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T12:59:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T12:59:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T12:59:24.515 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T12:59:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-23T12:59:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T13:00:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T13:00:00.816 INFO:teuthology.orchestra.run.smithi007.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-23T13:00:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:00:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-23T13:00:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-23T13:00:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-23T13:00:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T13:00:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T13:00:01.729 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T13:00:01.729 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:00:01.729 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-23T13:00:01.729 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-23T13:00:01.729 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-23T13:00:01.729 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-23T13:00:01.730 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-23T13:00:01.730 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:00:01.730 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-23T13:00:02.053 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T13:00:02.202 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-23T13:00:02.203 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-23T13:00:02.203 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-23T13:00:02.203 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-23T13:00:02.203 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-23T13:00:02.203 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-23T13:00:02.203 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-03-23T13:00:02.203 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T13:00:02.286 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-23T13:00:02.287 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-23T13:00:02.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-03-23T13:00:02.324 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-03-23T13:00:02.324 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-03-23T13:00:02.324 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-03-23T13:00:02.324 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 68.el9 2024-03-23T13:00:02.324 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-03-23T13:00:02.324 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-23T13:00:02.324 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-03-23T13:00:02.324 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8264254 2024-03-23T13:00:02.324 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-03-23T13:00:02.324 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:00:02.325 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-03-23T13:00:02.326 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-23T13:00:02.326 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-430.el9.x86_64 2024-03-23T13:00:02.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-23T13:00:02.949 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-03-23T13:00:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-23T13:00:04.473 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-03-23T13:00:04.476 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-03-23T13:00:04.491 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.RS55LB83nM 2024-03-23T13:00:04.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RS55LB83nM 2024-03-23T13:00:04.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.RS55LB83nM 2024-03-23T13:00:04.715 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.RS55LB83nM is 6KB 2024-03-23T13:00:04.763 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.RS55LB83nM 2024-03-23T13:00:04.778 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-03-23T13:00:04.846 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-23T13:00:04.846 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-23T13:00:04.846 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-23T13:00:04.846 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-23T13:00:04.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-23T13:00:04.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-03-23T13:00:35.001 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-23T13:00:35.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-23T13:00:35.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T13:00:53.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-03-23T13:01:02.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-23T13:01:02.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T13:02:02.410 DEBUG:teuthology.orchestra.remote:timed out 2024-03-23T13:02:14.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-23T13:02:14.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T13:02:14.704 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-03-23T13:02:15.099 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-23T13:02:15.100 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-430.el9.x86_64"... 2024-03-23T13:02:15.100 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-03-23T13:02:15.116 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-430.el9.x86_64 2024-03-23T13:02:15.116 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-23T13:02:15.116 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T13:02:15.116 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-23T13:02:16.117 DEBUG:teuthology.parallel:result is None 2024-03-23T13:02:16.118 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-23T13:02:16.125 INFO:teuthology.task.internal:Creating test directory... 2024-03-23T13:02:16.125 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T13:02:16.128 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T13:02:16.131 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T13:02:16.150 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-23T13:02:16.156 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-23T13:02:16.166 INFO:teuthology.task.internal:Creating archive directory... 2024-03-23T13:02:16.166 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T13:02:16.186 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T13:02:16.189 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T13:02:16.370 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-23T13:02:16.377 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-23T13:02:16.377 DEBUG:teuthology.orchestra.run.smithi130:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T13:02:16.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:02:16.393 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T13:02:16.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:02:16.410 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T13:02:16.412 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T13:02:16.436 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T13:02:16.476 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T13:02:16.500 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T13:02:16.505 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T13:02:16.511 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T13:02:16.529 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T13:02:16.544 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T13:02:16.546 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-23T13:02:16.553 INFO:teuthology.task.internal:Configuring sudo... 2024-03-23T13:02:16.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T13:02:16.555 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T13:02:16.558 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T13:02:16.587 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-23T13:02:16.596 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-23T13:02:16.596 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T13:02:16.626 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T13:02:16.628 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T13:02:16.644 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T13:02:16.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T13:02:16.772 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T13:02:16.858 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T13:02:16.902 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:02:16.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T13:02:16.972 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T13:02:17.014 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T13:02:17.071 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T13:02:17.155 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T13:02:17.198 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:02:17.198 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T13:02:17.267 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T13:02:17.312 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T13:02:17.370 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T13:02:17.455 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T13:02:17.497 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:02:17.497 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T13:02:17.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-03-23T13:02:17.568 DEBUG:teuthology.orchestra.run.smithi130:> sudo service rsyslog restart 2024-03-23T13:02:17.570 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-03-23T13:02:17.603 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T13:02:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T13:02:17.639 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T13:02:18.086 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-23T13:02:18.093 INFO:teuthology.task.internal:Starting timer... 2024-03-23T13:02:18.094 INFO:teuthology.run_tasks:Running task pcp... 2024-03-23T13:02:18.106 INFO:teuthology.run_tasks:Running task selinux... 2024-03-23T13:02:18.114 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-23T13:02:18.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-03-23T13:02:18.169 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-03-23T13:02:18.171 DEBUG:teuthology.orchestra.run.smithi130:> sudo service auditd rotate 2024-03-23T13:02:18.217 INFO:teuthology.orchestra.run.smithi130.stdout:Rotating logs: 2024-03-23T13:02:18.219 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-03-23T13:02:18.273 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-03-23T13:02:18.275 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-23T13:02:18.275 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-03-23T13:02:18.304 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-03-23T13:02:18.306 DEBUG:teuthology.orchestra.run.smithi130:> /usr/sbin/getenforce 2024-03-23T13:02:18.325 INFO:teuthology.orchestra.run.smithi130.stdout:Permissive 2024-03-23T13:02:18.325 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-03-23T13:02:18.343 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-03-23T13:02:18.343 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi130.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2024-03-23T13:02:18.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T13:02:18.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:02:18.374 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T13:02:18.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:02:18.402 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T13:02:18.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:02:18.430 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-23T13:02:18.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-03-23T13:02:18.465 DEBUG:teuthology.orchestra.run.smithi130:> sudo /usr/sbin/setenforce permissive 2024-03-23T13:02:18.502 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-03-23T13:02:18.536 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-23T13:02:18.546 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-23T13:02:18.551 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-23T13:02:18.681 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-23T13:02:18.697 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-23T13:02:18.698 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi130.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-23T13:06:45.081 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi130.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-03-23T13:06:45.082 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-23T13:06:45.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T13:06:45.173 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-03-23T13:06:45.252 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-23T13:06:45.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-03-23T13:06:45.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T13:06:45.340 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-03-23T13:06:45.408 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-03-23T13:06:45.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-23T13:06:45.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T13:06:45.493 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-03-23T13:06:45.561 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-23T13:06:45.561 INFO:teuthology.run_tasks:Running task clock... 2024-03-23T13:06:45.571 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-23T13:06:45.571 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T13:06:45.571 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T13:06:45.574 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T13:06:45.574 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T13:06:45.576 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T13:06:45.576 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T13:06:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T13:06:45.605 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T13:06:45.619 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T13:06:45.622 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T13:06:45.632 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T13:06:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-03-23T13:06:45.649 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T13:06:45.650 INFO:teuthology.orchestra.run.smithi130.stderr:sudo: ntpd: command not found 2024-03-23T13:06:45.662 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-03-23T13:06:45.664 INFO:teuthology.orchestra.run.smithi130.stdout:506 Cannot talk to daemon 2024-03-23T13:06:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T13:06:45.679 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-03-23T13:06:45.680 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T13:06:45.692 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-03-23T13:06:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T13:06:45.697 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T13:06:45.708 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T13:06:45.724 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T13:06:45.740 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-03-23T13:06:45.744 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T13:06:45.744 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-03-23T13:06:45.744 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.744 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.744 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.745 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.751 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-03-23T13:06:45.754 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T13:06:45.754 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-03-23T13:06:45.754 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.754 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.754 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.755 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.768 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-03-23T13:06:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T13:06:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-03-23T13:06:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:06:45.773 INFO:teuthology.run_tasks:Running task install... 2024-03-23T13:06:45.782 DEBUG:teuthology.task.install:project ceph 2024-03-23T13:06:45.782 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'}} 2024-03-23T13:06:45.782 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'} 2024-03-23T13:06:45.782 INFO:teuthology.task.install:Using flavor: default 2024-03-23T13:06:45.789 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-23T13:06:45.789 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-23T13:06:45.789 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'tag': None, 'wait_for_package': False} 2024-03-23T13:06:45.790 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:06:45.791 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'tag': None, 'wait_for_package': False} 2024-03-23T13:06:45.792 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:06:45.793 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'tag': None, 'wait_for_package': False} 2024-03-23T13:06:45.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:06:45.940 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/ 2024-03-23T13:06:45.940 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1855.g21b3bd67 2024-03-23T13:06:46.007 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/ 2024-03-23T13:06:46.007 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1855.g21b3bd67 2024-03-23T13:06:46.014 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/ 2024-03-23T13:06:46.014 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1855.g21b3bd67 2024-03-23T13:06:46.114 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T13:06:46.114 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:06:46.114 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T13:06:46.120 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T13:06:46.120 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:06:46.120 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T13:06:46.142 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T13:06:46.142 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/21b3bd672163092b4e542f772797fcba02d3d4de/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T13:06:46.148 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T13:06:46.149 DEBUG:teuthology.orchestra.run.smithi130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/21b3bd672163092b4e542f772797fcba02d3d4de/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T13:06:46.176 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T13:06:46.176 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:06:46.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T13:06:46.205 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T13:06:46.205 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/21b3bd672163092b4e542f772797fcba02d3d4de/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T13:06:46.210 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T13:06:46.216 DEBUG:teuthology.orchestra.run.smithi130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T13:06:46.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T13:06:46.289 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T13:06:46.297 DEBUG:teuthology.orchestra.run.smithi130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T13:06:46.352 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T13:06:46.356 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-03-23T13:06:46.358 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-03-23T13:06:46.363 INFO:teuthology.orchestra.run.smithi130.stdout:check_obsoletes = 1 2024-03-23T13:06:46.364 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum clean all 2024-03-23T13:06:46.418 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-03-23T13:06:46.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-03-23T13:06:46.593 INFO:teuthology.orchestra.run.smithi155.stdout:74 files removed 2024-03-23T13:06:46.598 INFO:teuthology.orchestra.run.smithi130.stdout:74 files removed 2024-03-23T13:06:46.620 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T13:06:46.625 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T13:06:46.660 INFO:teuthology.orchestra.run.smithi007.stdout:74 files removed 2024-03-23T13:06:46.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T13:06:47.285 INFO:teuthology.orchestra.run.smithi130.stdout:ceph packages for x86_64 224 kB/s | 83 kB 00:00 2024-03-23T13:06:47.290 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 219 kB/s | 83 kB 00:00 2024-03-23T13:06:47.346 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 235 kB/s | 83 kB 00:00 2024-03-23T13:06:47.635 INFO:teuthology.orchestra.run.smithi130.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-23T13:06:47.699 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-23T13:06:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-23T13:06:47.986 INFO:teuthology.orchestra.run.smithi130.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-03-23T13:06:48.188 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-23T13:06:48.388 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-23T13:06:48.590 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-23T13:06:49.755 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2024-03-23T13:06:50.049 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2024-03-23T13:06:50.655 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-23T13:06:51.565 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-03-23T13:06:52.748 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-03-23T13:06:53.941 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-03-23T13:06:54.225 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 299 B/s | 1.9 kB 00:06 2024-03-23T13:06:54.585 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - AppStream 20 MB/s | 6.7 MB 00:00 2024-03-23T13:06:56.077 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-23T13:06:58.018 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-03-23T13:06:58.092 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-03-23T13:06:58.448 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.1 MB 00:02 2024-03-23T13:06:59.743 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-03-23T13:06:59.882 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-03-23T13:07:00.617 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 33 MB/s | 21 MB 00:00 2024-03-23T13:07:00.819 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-03-23T13:07:01.522 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-23T13:07:05.634 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 6.8 kB/s | 1.7 kB 00:00 2024-03-23T13:07:06.619 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-03-23T13:07:07.861 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.1 MB 00:03 2024-03-23T13:07:07.940 INFO:teuthology.orchestra.run.smithi130.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T13:07:07.940 INFO:teuthology.orchestra.run.smithi130.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T13:07:07.954 INFO:teuthology.orchestra.run.smithi130.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T13:07:07.956 INFO:teuthology.orchestra.run.smithi130.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T13:07:07.957 INFO:teuthology.orchestra.run.smithi130.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T13:07:07.958 INFO:teuthology.orchestra.run.smithi130.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T13:07:07.960 INFO:teuthology.orchestra.run.smithi130.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T13:07:07.963 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T13:07:07.964 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T13:07:08.043 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-03-23T13:07:08.055 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================================ 2024-03-23T13:07:08.055 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-03-23T13:07:08.056 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================================ 2024-03-23T13:07:08.056 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-03-23T13:07:08.056 INFO:teuthology.orchestra.run.smithi130.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T13:07:08.056 INFO:teuthology.orchestra.run.smithi130.stdout: ceph x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 6.6 k 2024-03-23T13:07:08.056 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.3 M 2024-03-23T13:07:08.056 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 888 k 2024-03-23T13:07:08.057 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 145 k 2024-03-23T13:07:08.057 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 1.5 M 2024-03-23T13:07:08.057 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 138 k 2024-03-23T13:07:08.057 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 1.8 M 2024-03-23T13:07:08.057 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 7.4 M 2024-03-23T13:07:08.057 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 49 k 2024-03-23T13:07:08.057 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 9.8 M 2024-03-23T13:07:08.058 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 47 M 2024-03-23T13:07:08.058 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 264 k 2024-03-23T13:07:08.058 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 748 k 2024-03-23T13:07:08.058 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 219 k 2024-03-23T13:07:08.058 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-shell x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 27 k 2024-03-23T13:07:08.058 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 18 k 2024-03-23T13:07:08.059 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T13:07:08.059 INFO:teuthology.orchestra.run.smithi130.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T13:07:08.059 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T13:07:08.059 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 32 k 2024-03-23T13:07:08.059 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 744 k 2024-03-23T13:07:08.059 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 127 k 2024-03-23T13:07:08.059 INFO:teuthology.orchestra.run.smithi130.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T13:07:08.060 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T13:07:08.060 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T13:07:08.060 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T13:07:08.060 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T13:07:08.060 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T13:07:08.060 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 161 k 2024-03-23T13:07:08.061 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 320 k 2024-03-23T13:07:08.061 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 298 k 2024-03-23T13:07:08.061 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 99 k 2024-03-23T13:07:08.061 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 85 k 2024-03-23T13:07:08.061 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T13:07:08.061 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 170 k 2024-03-23T13:07:08.061 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T13:07:08.062 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrading: 2024-03-23T13:07:08.062 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T13:07:08.062 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T13:07:08.062 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T13:07:08.062 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T13:07:08.063 INFO:teuthology.orchestra.run.smithi130.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T13:07:08.063 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T13:07:08.063 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T13:07:08.063 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T13:07:08.063 INFO:teuthology.orchestra.run.smithi130.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T13:07:08.063 INFO:teuthology.orchestra.run.smithi130.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T13:07:08.063 INFO:teuthology.orchestra.run.smithi130.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T13:07:08.064 INFO:teuthology.orchestra.run.smithi130.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T13:07:08.064 INFO:teuthology.orchestra.run.smithi130.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T13:07:08.064 INFO:teuthology.orchestra.run.smithi130.stdout: librados2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.4 M 2024-03-23T13:07:08.064 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T13:07:08.064 INFO:teuthology.orchestra.run.smithi130.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T13:07:08.064 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T13:07:08.065 INFO:teuthology.orchestra.run.smithi130.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T13:07:08.065 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T13:07:08.065 INFO:teuthology.orchestra.run.smithi130.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T13:07:08.065 INFO:teuthology.orchestra.run.smithi130.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T13:07:08.065 INFO:teuthology.orchestra.run.smithi130.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T13:07:08.065 INFO:teuthology.orchestra.run.smithi130.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T13:07:08.065 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T13:07:08.066 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-03-23T13:07:08.066 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-23T13:07:08.066 INFO:teuthology.orchestra.run.smithi130.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T13:07:08.066 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T13:07:08.066 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 21 M 2024-03-23T13:07:08.066 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 31 k 2024-03-23T13:07:08.067 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 2.4 M 2024-03-23T13:07:08.067 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 248 k 2024-03-23T13:07:08.067 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 4.7 M 2024-03-23T13:07:08.067 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 17 M 2024-03-23T13:07:08.067 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 15 k 2024-03-23T13:07:08.067 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 25 k 2024-03-23T13:07:08.068 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T13:07:08.068 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T13:07:08.068 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T13:07:08.068 INFO:teuthology.orchestra.run.smithi130.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T13:07:08.068 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T13:07:08.068 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-23T13:07:08.068 INFO:teuthology.orchestra.run.smithi130.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T13:07:08.069 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T13:07:08.069 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T13:07:08.069 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 163 k 2024-03-23T13:07:08.069 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T13:07:08.069 INFO:teuthology.orchestra.run.smithi130.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T13:07:08.069 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T13:07:08.070 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T13:07:08.070 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T13:07:08.070 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 499 k 2024-03-23T13:07:08.070 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T13:07:08.070 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.1 M 2024-03-23T13:07:08.070 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T13:07:08.071 INFO:teuthology.orchestra.run.smithi130.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T13:07:08.071 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T13:07:08.071 INFO:teuthology.orchestra.run.smithi130.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-23T13:07:08.071 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-23T13:07:08.071 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-23T13:07:08.071 INFO:teuthology.orchestra.run.smithi130.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T13:07:08.071 INFO:teuthology.orchestra.run.smithi130.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T13:07:08.072 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T13:07:08.072 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T13:07:08.072 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T13:07:08.072 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-23T13:07:08.072 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-23T13:07:08.072 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T13:07:08.073 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-23T13:07:08.073 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T13:07:08.073 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T13:07:08.073 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T13:07:08.073 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 45 k 2024-03-23T13:07:08.073 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 131 k 2024-03-23T13:07:08.074 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T13:07:08.074 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T13:07:08.074 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T13:07:08.074 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T13:07:08.074 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-23T13:07:08.075 INFO:teuthology.orchestra.run.smithi130.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-23T13:07:08.075 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T13:07:08.075 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T13:07:08.075 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T13:07:08.076 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-23T13:07:08.076 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-23T13:07:08.076 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T13:07:08.076 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T13:07:08.076 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T13:07:08.077 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T13:07:08.077 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T13:07:08.077 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T13:07:08.077 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T13:07:08.077 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T13:07:08.078 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T13:07:08.078 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T13:07:08.078 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T13:07:08.078 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T13:07:08.079 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T13:07:08.079 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T13:07:08.079 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T13:07:08.079 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T13:07:08.079 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T13:07:08.080 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T13:07:08.080 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T13:07:08.080 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T13:07:08.080 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T13:07:08.080 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T13:07:08.081 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-23T13:07:08.081 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T13:07:08.081 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T13:07:08.081 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T13:07:08.081 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T13:07:08.081 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-23T13:07:08.082 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T13:07:08.082 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T13:07:08.082 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T13:07:08.082 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T13:07:08.083 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T13:07:08.083 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T13:07:08.083 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T13:07:08.083 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T13:07:08.083 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T13:07:08.084 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T13:07:08.084 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T13:07:08.084 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-23T13:07:08.084 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T13:07:08.084 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T13:07:08.085 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T13:07:08.085 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T13:07:08.085 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-23T13:07:08.085 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-23T13:07:08.085 INFO:teuthology.orchestra.run.smithi130.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T13:07:08.085 INFO:teuthology.orchestra.run.smithi130.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T13:07:08.085 INFO:teuthology.orchestra.run.smithi130.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T13:07:08.086 INFO:teuthology.orchestra.run.smithi130.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T13:07:08.086 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T13:07:08.086 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T13:07:08.086 INFO:teuthology.orchestra.run.smithi130.stdout:Installing weak dependencies: 2024-03-23T13:07:08.086 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-23T13:07:08.086 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:07:08.087 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-03-23T13:07:08.087 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================================ 2024-03-23T13:07:08.087 INFO:teuthology.orchestra.run.smithi130.stdout:Install 150 Packages 2024-03-23T13:07:08.087 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrade 24 Packages 2024-03-23T13:07:08.087 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:07:08.088 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 224 M 2024-03-23T13:07:08.088 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-03-23T13:07:08.544 INFO:teuthology.orchestra.run.smithi130.stdout:(1/174): ceph-19.0.0-1855.g21b3bd67.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-03-23T13:07:08.836 INFO:teuthology.orchestra.run.smithi130.stdout:(2/174): ceph-fuse-19.0.0-1855.g21b3bd67.el9.x8 3.0 MB/s | 888 kB 00:00 2024-03-23T13:07:08.911 INFO:teuthology.orchestra.run.smithi130.stdout:(3/174): ceph-immutable-object-cache-19.0.0-185 1.9 MB/s | 145 kB 00:00 2024-03-23T13:07:09.102 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T13:07:09.102 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T13:07:09.118 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T13:07:09.119 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T13:07:09.120 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T13:07:09.122 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T13:07:09.124 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T13:07:09.127 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T13:07:09.128 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T13:07:09.216 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-23T13:07:09.228 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================================ 2024-03-23T13:07:09.228 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================================ 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 6.6 k 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.3 M 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 888 k 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 145 k 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 1.5 M 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 138 k 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 1.8 M 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 7.4 M 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 49 k 2024-03-23T13:07:09.229 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 9.8 M 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 47 M 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 264 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 748 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 219 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 27 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 18 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 32 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 744 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 127 k 2024-03-23T13:07:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 161 k 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 320 k 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 298 k 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 99 k 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 85 k 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 170 k 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T13:07:09.231 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T13:07:09.232 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.4 M 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T13:07:09.233 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 21 M 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 31 k 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 2.4 M 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 248 k 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 4.7 M 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 17 M 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 15 k 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 25 k 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T13:07:09.234 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 163 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 499 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T13:07:09.235 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.1 M 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-23T13:07:09.236 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 45 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 131 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-23T13:07:09.237 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T13:07:09.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T13:07:09.239 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T13:07:09.240 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T13:07:09.241 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T13:07:09.242 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T13:07:09.242 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-03-23T13:07:09.242 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-23T13:07:09.242 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:07:09.242 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-23T13:07:09.242 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================================ 2024-03-23T13:07:09.242 INFO:teuthology.orchestra.run.smithi007.stdout:Install 150 Packages 2024-03-23T13:07:09.242 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 24 Packages 2024-03-23T13:07:09.242 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:07:09.243 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 224 M 2024-03-23T13:07:09.243 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-23T13:07:09.445 INFO:teuthology.orchestra.run.smithi130.stdout:(4/174): ceph-mds-19.0.0-1855.g21b3bd67.el9.x86 4.4 MB/s | 2.4 MB 00:00 2024-03-23T13:07:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:(5/174): ceph-base-19.0.0-1855.g21b3bd67.el9.x8 4.3 MB/s | 5.3 MB 00:01 2024-03-23T13:07:09.579 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-03-23T13:07:09.970 INFO:teuthology.orchestra.run.smithi130.stdout:(6/174): ceph-common-19.0.0-1855.g21b3bd67.el9. 13 MB/s | 21 MB 00:01 2024-03-23T13:07:10.004 INFO:teuthology.orchestra.run.smithi130.stdout:(7/174): ceph-mgr-19.0.0-1855.g21b3bd67.el9.x86 2.7 MB/s | 1.5 MB 00:00 2024-03-23T13:07:10.346 INFO:teuthology.orchestra.run.smithi007.stdout:(1/174): ceph-19.0.0-1855.g21b3bd67.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-03-23T13:07:10.354 INFO:teuthology.orchestra.run.smithi130.stdout:(8/174): ceph-mon-19.0.0-1855.g21b3bd67.el9.x86 6.0 MB/s | 4.7 MB 00:00 2024-03-23T13:07:10.438 INFO:teuthology.orchestra.run.smithi130.stdout:(9/174): ceph-selinux-19.0.0-1855.g21b3bd67.el9 303 kB/s | 25 kB 00:00 2024-03-23T13:07:11.271 INFO:teuthology.orchestra.run.smithi007.stdout:(2/174): ceph-fuse-19.0.0-1855.g21b3bd67.el9.x8 960 kB/s | 888 kB 00:00 2024-03-23T13:07:11.346 INFO:teuthology.orchestra.run.smithi130.stdout:(10/174): ceph-osd-19.0.0-1855.g21b3bd67.el9.x8 12 MB/s | 17 MB 00:01 2024-03-23T13:07:11.363 INFO:teuthology.orchestra.run.smithi007.stdout:(3/174): ceph-immutable-object-cache-19.0.0-185 1.5 MB/s | 145 kB 00:00 2024-03-23T13:07:11.430 INFO:teuthology.orchestra.run.smithi130.stdout:(11/174): cephfs-mirror-19.0.0-1855.g21b3bd67.e 2.6 MB/s | 219 kB 00:00 2024-03-23T13:07:11.505 INFO:teuthology.orchestra.run.smithi130.stdout:(12/174): cephfs-shell-19.0.0-1855.g21b3bd67.el 361 kB/s | 27 kB 00:00 2024-03-23T13:07:11.580 INFO:teuthology.orchestra.run.smithi130.stdout:(13/174): libcephfs-devel-19.0.0-1855.g21b3bd67 425 kB/s | 32 kB 00:00 2024-03-23T13:07:11.864 INFO:teuthology.orchestra.run.smithi130.stdout:(14/174): ceph-radosgw-19.0.0-1855.g21b3bd67.el 5.3 MB/s | 9.8 MB 00:01 2024-03-23T13:07:11.931 INFO:teuthology.orchestra.run.smithi130.stdout:(15/174): libcephfs2-19.0.0-1855.g21b3bd67.el9. 2.1 MB/s | 744 kB 00:00 2024-03-23T13:07:11.956 INFO:teuthology.orchestra.run.smithi130.stdout:(16/174): libcephsqlite-19.0.0-1855.g21b3bd67.e 1.7 MB/s | 163 kB 00:00 2024-03-23T13:07:11.998 INFO:teuthology.orchestra.run.smithi130.stdout:(17/174): librados-devel-19.0.0-1855.g21b3bd67. 1.9 MB/s | 127 kB 00:00 2024-03-23T13:07:12.123 INFO:teuthology.orchestra.run.smithi130.stdout:(18/174): libradosstriper1-19.0.0-1855.g21b3bd6 2.9 MB/s | 499 kB 00:00 2024-03-23T13:07:12.224 INFO:teuthology.orchestra.run.smithi130.stdout:(19/174): python3-ceph-argparse-19.0.0-1855.g21 451 kB/s | 45 kB 00:00 2024-03-23T13:07:12.324 INFO:teuthology.orchestra.run.smithi130.stdout:(20/174): python3-ceph-common-19.0.0-1855.g21b3 1.3 MB/s | 131 kB 00:00 2024-03-23T13:07:12.349 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 21 MB 00:02 2024-03-23T13:07:12.432 INFO:teuthology.orchestra.run.smithi130.stdout:(21/174): python3-cephfs-19.0.0-1855.g21b3bd67. 1.5 MB/s | 161 kB 00:00 2024-03-23T13:07:12.541 INFO:teuthology.orchestra.run.smithi130.stdout:(22/174): python3-rados-19.0.0-1855.g21b3bd67.e 2.9 MB/s | 320 kB 00:00 2024-03-23T13:07:12.658 INFO:teuthology.orchestra.run.smithi130.stdout:(23/174): python3-rbd-19.0.0-1855.g21b3bd67.el9 2.5 MB/s | 298 kB 00:00 2024-03-23T13:07:12.733 INFO:teuthology.orchestra.run.smithi130.stdout:(24/174): librgw2-19.0.0-1855.g21b3bd67.el9.x86 7.0 MB/s | 5.1 MB 00:00 2024-03-23T13:07:12.746 INFO:teuthology.orchestra.run.smithi007.stdout:(4/174): ceph-mds-19.0.0-1855.g21b3bd67.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-03-23T13:07:12.750 INFO:teuthology.orchestra.run.smithi130.stdout:(25/174): python3-rgw-19.0.0-1855.g21b3bd67.el9 1.1 MB/s | 99 kB 00:00 2024-03-23T13:07:12.784 INFO:teuthology.orchestra.run.smithi130.stdout:(26/174): rbd-fuse-19.0.0-1855.g21b3bd67.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-23T13:07:12.838 INFO:teuthology.orchestra.run.smithi007.stdout:(5/174): ceph-base-19.0.0-1855.g21b3bd67.el9.x8 1.9 MB/s | 5.3 MB 00:02 2024-03-23T13:07:12.851 INFO:teuthology.orchestra.run.smithi130.stdout:(27/174): rbd-nbd-19.0.0-1855.g21b3bd67.el9.x86 2.5 MB/s | 170 kB 00:00 2024-03-23T13:07:12.943 INFO:teuthology.orchestra.run.smithi130.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1855.g 343 kB/s | 31 kB 00:00 2024-03-23T13:07:13.035 INFO:teuthology.orchestra.run.smithi130.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1855.g21b3bd6 1.5 MB/s | 138 kB 00:00 2024-03-23T13:07:13.377 INFO:teuthology.orchestra.run.smithi130.stdout:(30/174): ceph-mgr-dashboard-19.0.0-1855.g21b3b 5.2 MB/s | 1.8 MB 00:00 2024-03-23T13:07:13.560 INFO:teuthology.orchestra.run.smithi130.stdout:(31/174): rbd-mirror-19.0.0-1855.g21b3bd67.el9. 3.8 MB/s | 3.1 MB 00:00 2024-03-23T13:07:13.613 INFO:teuthology.orchestra.run.smithi007.stdout:(6/174): ceph-mgr-19.0.0-1855.g21b3bd67.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-03-23T13:07:13.677 INFO:teuthology.orchestra.run.smithi130.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1855.g21 2.1 MB/s | 248 kB 00:00 2024-03-23T13:07:13.747 INFO:teuthology.orchestra.run.smithi007.stdout:(7/174): ceph-mon-19.0.0-1855.g21b3bd67.el9.x86 5.2 MB/s | 4.7 MB 00:00 2024-03-23T13:07:13.761 INFO:teuthology.orchestra.run.smithi130.stdout:(33/174): ceph-mgr-rook-19.0.0-1855.g21b3bd67.e 593 kB/s | 49 kB 00:00 2024-03-23T13:07:13.861 INFO:teuthology.orchestra.run.smithi130.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1855.g2 153 kB/s | 15 kB 00:00 2024-03-23T13:07:13.970 INFO:teuthology.orchestra.run.smithi130.stdout:(35/174): ceph-volume-19.0.0-1855.g21b3bd67.el9 2.4 MB/s | 264 kB 00:00 2024-03-23T13:07:14.570 INFO:teuthology.orchestra.run.smithi130.stdout:(36/174): ceph-test-19.0.0-1855.g21b3bd67.el9.x 11 MB/s | 47 MB 00:04 2024-03-23T13:07:14.604 INFO:teuthology.orchestra.run.smithi130.stdout:(37/174): cephadm-19.0.0-1855.g21b3bd67.el9.noa 1.2 MB/s | 748 kB 00:00 2024-03-23T13:07:14.629 INFO:teuthology.orchestra.run.smithi130.stdout:(38/174): cephfs-top-19.0.0-1855.g21b3bd67.el9. 313 kB/s | 18 kB 00:00 2024-03-23T13:07:14.689 INFO:teuthology.orchestra.run.smithi007.stdout:(8/174): ceph-radosgw-19.0.0-1855.g21b3bd67.el9 10 MB/s | 9.8 MB 00:00 2024-03-23T13:07:14.746 INFO:teuthology.orchestra.run.smithi130.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-03-23T13:07:14.748 INFO:teuthology.orchestra.run.smithi007.stdout:(9/174): ceph-selinux-19.0.0-1855.g21b3bd67.el9 433 kB/s | 25 kB 00:00 2024-03-23T13:07:14.854 INFO:teuthology.orchestra.run.smithi130.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 662 kB 00:00 2024-03-23T13:07:14.880 INFO:teuthology.orchestra.run.smithi130.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 992 kB/s | 106 kB 00:00 2024-03-23T13:07:14.913 INFO:teuthology.orchestra.run.smithi130.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 920 kB 00:00 2024-03-23T13:07:14.938 INFO:teuthology.orchestra.run.smithi130.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-03-23T13:07:14.964 INFO:teuthology.orchestra.run.smithi130.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 382 kB/s | 32 kB 00:00 2024-03-23T13:07:15.006 INFO:teuthology.orchestra.run.smithi130.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 257 kB/s | 17 kB 00:00 2024-03-23T13:07:15.073 INFO:teuthology.orchestra.run.smithi130.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-03-23T13:07:15.089 INFO:teuthology.orchestra.run.smithi130.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 80 kB/s | 10 kB 00:00 2024-03-23T13:07:15.115 INFO:teuthology.orchestra.run.smithi130.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 159 kB 00:00 2024-03-23T13:07:15.131 INFO:teuthology.orchestra.run.smithi130.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 759 kB/s | 44 kB 00:00 2024-03-23T13:07:15.165 INFO:teuthology.orchestra.run.smithi130.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-23T13:07:15.190 INFO:teuthology.orchestra.run.smithi130.stdout:(51/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-23T13:07:15.216 INFO:teuthology.orchestra.run.smithi130.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 554 kB/s | 46 kB 00:00 2024-03-23T13:07:15.241 INFO:teuthology.orchestra.run.smithi130.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-23T13:07:15.266 INFO:teuthology.orchestra.run.smithi130.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-23T13:07:15.291 INFO:teuthology.orchestra.run.smithi130.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-23T13:07:15.317 INFO:teuthology.orchestra.run.smithi130.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 455 kB/s | 34 kB 00:00 2024-03-23T13:07:15.359 INFO:teuthology.orchestra.run.smithi130.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 23 MB/s | 1.5 MB 00:00 2024-03-23T13:07:15.467 INFO:teuthology.orchestra.run.smithi130.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-03-23T13:07:15.509 INFO:teuthology.orchestra.run.smithi130.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 4.2 MB/s | 811 kB 00:00 2024-03-23T13:07:15.535 INFO:teuthology.orchestra.run.smithi130.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 774 kB/s | 134 kB 00:00 2024-03-23T13:07:15.640 INFO:teuthology.orchestra.run.smithi130.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2024-03-23T13:07:15.665 INFO:teuthology.orchestra.run.smithi130.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 607 kB/s | 78 kB 00:00 2024-03-23T13:07:15.699 INFO:teuthology.orchestra.run.smithi130.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.2 MB/s | 1.0 MB 00:00 2024-03-23T13:07:15.799 INFO:teuthology.orchestra.run.smithi130.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 37 MB/s | 5.8 MB 00:00 2024-03-23T13:07:15.825 INFO:teuthology.orchestra.run.smithi130.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 157 kB/s | 24 kB 00:00 2024-03-23T13:07:15.850 INFO:teuthology.orchestra.run.smithi130.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 122 kB/s | 18 kB 00:00 2024-03-23T13:07:15.875 INFO:teuthology.orchestra.run.smithi130.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-23T13:07:15.901 INFO:teuthology.orchestra.run.smithi130.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-03-23T13:07:15.993 INFO:teuthology.orchestra.run.smithi130.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-23T13:07:16.018 INFO:teuthology.orchestra.run.smithi130.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-23T13:07:16.043 INFO:teuthology.orchestra.run.smithi130.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-23T13:07:16.068 INFO:teuthology.orchestra.run.smithi130.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 925 kB/s | 69 kB 00:00 2024-03-23T13:07:16.094 INFO:teuthology.orchestra.run.smithi130.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.7 MB/s | 238 kB 00:00 2024-03-23T13:07:16.119 INFO:teuthology.orchestra.run.smithi130.stdout:(74/174): python3-prettytable-0.7.2-27.el9.noar 546 kB/s | 41 kB 00:00 2024-03-23T13:07:16.144 INFO:teuthology.orchestra.run.smithi130.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-23T13:07:16.169 INFO:teuthology.orchestra.run.smithi130.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-23T13:07:16.194 INFO:teuthology.orchestra.run.smithi130.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 865 kB/s | 43 kB 00:00 2024-03-23T13:07:16.220 INFO:teuthology.orchestra.run.smithi130.stdout:(78/174): python3-pytz-2021.1-5.el9.noarch.rpm 633 kB/s | 47 kB 00:00 2024-03-23T13:07:16.387 INFO:teuthology.orchestra.run.smithi130.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 204 kB/s | 39 kB 00:00 2024-03-23T13:07:16.546 INFO:teuthology.orchestra.run.smithi130.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-03-23T13:07:16.571 INFO:teuthology.orchestra.run.smithi130.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 118 kB/s | 41 kB 00:00 2024-03-23T13:07:16.596 INFO:teuthology.orchestra.run.smithi130.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-03-23T13:07:16.621 INFO:teuthology.orchestra.run.smithi130.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-23T13:07:16.647 INFO:teuthology.orchestra.run.smithi130.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 838 kB/s | 63 kB 00:00 2024-03-23T13:07:16.672 INFO:teuthology.orchestra.run.smithi130.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 585 kB/s | 44 kB 00:00 2024-03-23T13:07:16.697 INFO:teuthology.orchestra.run.smithi130.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-23T13:07:16.722 INFO:teuthology.orchestra.run.smithi130.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-23T13:07:16.764 INFO:teuthology.orchestra.run.smithi130.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-23T13:07:16.789 INFO:teuthology.orchestra.run.smithi130.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-23T13:07:16.815 INFO:teuthology.orchestra.run.smithi130.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-23T13:07:16.840 INFO:teuthology.orchestra.run.smithi130.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-23T13:07:16.982 INFO:teuthology.orchestra.run.smithi130.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-23T13:07:17.007 INFO:teuthology.orchestra.run.smithi130.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-03-23T13:07:17.049 INFO:teuthology.orchestra.run.smithi130.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-23T13:07:17.099 INFO:teuthology.orchestra.run.smithi130.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-03-23T13:07:17.206 INFO:teuthology.orchestra.run.smithi007.stdout:(10/174): ceph-test-19.0.0-1855.g21b3bd67.el9.x 19 MB/s | 47 MB 00:02 2024-03-23T13:07:17.255 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-23T13:07:17.258 INFO:teuthology.orchestra.run.smithi130.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 86 kB/s | 21 kB 00:00 2024-03-23T13:07:17.264 INFO:teuthology.orchestra.run.smithi007.stdout:(11/174): cephfs-mirror-19.0.0-1855.g21b3bd67.e 3.7 MB/s | 219 kB 00:00 2024-03-23T13:07:17.300 INFO:teuthology.orchestra.run.smithi130.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 102 kB/s | 25 kB 00:00 2024-03-23T13:07:17.323 INFO:teuthology.orchestra.run.smithi007.stdout:(12/174): cephfs-shell-19.0.0-1855.g21b3bd67.el 464 kB/s | 27 kB 00:00 2024-03-23T13:07:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:(13/174): libcephfs-devel-19.0.0-1855.g21b3bd67 545 kB/s | 32 kB 00:00 2024-03-23T13:07:17.392 INFO:teuthology.orchestra.run.smithi130.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-03-23T13:07:17.417 INFO:teuthology.orchestra.run.smithi130.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-03-23T13:07:17.459 INFO:teuthology.orchestra.run.smithi130.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 216 kB 00:00 2024-03-23T13:07:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:(14/174): libcephfs2-19.0.0-1855.g21b3bd67.el9. 7.9 MB/s | 744 kB 00:00 2024-03-23T13:07:17.476 INFO:teuthology.orchestra.run.smithi130.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 598 kB/s | 35 kB 00:00 2024-03-23T13:07:17.541 INFO:teuthology.orchestra.run.smithi007.stdout:(15/174): libcephsqlite-19.0.0-1855.g21b3bd67.e 2.4 MB/s | 163 kB 00:00 2024-03-23T13:07:17.551 INFO:teuthology.orchestra.run.smithi130.stdout:(102/174): openssl-devel-3.0.7-27.el9.x86_64.rp 9.2 MB/s | 4.1 MB 00:00 2024-03-23T13:07:17.568 INFO:teuthology.orchestra.run.smithi130.stdout:(103/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 409 kB/s | 44 kB 00:00 2024-03-23T13:07:17.593 INFO:teuthology.orchestra.run.smithi130.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 798 kB/s | 94 kB 00:00 2024-03-23T13:07:17.607 INFO:teuthology.orchestra.run.smithi007.stdout:(16/174): librados-devel-19.0.0-1855.g21b3bd67. 1.9 MB/s | 127 kB 00:00 2024-03-23T13:07:17.699 INFO:teuthology.orchestra.run.smithi007.stdout:(17/174): libradosstriper1-19.0.0-1855.g21b3bd6 5.3 MB/s | 499 kB 00:00 2024-03-23T13:07:18.002 INFO:teuthology.orchestra.run.smithi130.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-03-23T13:07:18.016 INFO:teuthology.orchestra.run.smithi007.stdout:(18/174): librgw2-19.0.0-1855.g21b3bd67.el9.x86 16 MB/s | 5.1 MB 00:00 2024-03-23T13:07:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:(19/174): python3-ceph-argparse-19.0.0-1855.g21 776 kB/s | 45 kB 00:00 2024-03-23T13:07:18.094 INFO:teuthology.orchestra.run.smithi130.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-03-23T13:07:18.119 INFO:teuthology.orchestra.run.smithi130.stdout:(107/174): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-03-23T13:07:18.142 INFO:teuthology.orchestra.run.smithi007.stdout:(20/174): python3-ceph-common-19.0.0-1855.g21b3 1.9 MB/s | 131 kB 00:00 2024-03-23T13:07:18.161 INFO:teuthology.orchestra.run.smithi130.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 294 kB/s | 19 kB 00:00 2024-03-23T13:07:18.209 INFO:teuthology.orchestra.run.smithi007.stdout:(21/174): python3-cephfs-19.0.0-1855.g21b3bd67. 2.4 MB/s | 161 kB 00:00 2024-03-23T13:07:18.228 INFO:teuthology.orchestra.run.smithi130.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-03-23T13:07:18.284 INFO:teuthology.orchestra.run.smithi007.stdout:(22/174): python3-rados-19.0.0-1855.g21b3bd67.e 4.2 MB/s | 320 kB 00:00 2024-03-23T13:07:18.295 INFO:teuthology.orchestra.run.smithi130.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 729 kB/s | 49 kB 00:00 2024-03-23T13:07:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:(23/174): python3-rbd-19.0.0-1855.g21b3bd67.el9 3.9 MB/s | 298 kB 00:00 2024-03-23T13:07:18.395 INFO:teuthology.orchestra.run.smithi130.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-03-23T13:07:18.471 INFO:teuthology.orchestra.run.smithi130.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-23T13:07:18.560 INFO:teuthology.orchestra.run.smithi007.stdout:(24/174): ceph-common-19.0.0-1855.g21b3bd67.el9 2.4 MB/s | 21 MB 00:08 2024-03-23T13:07:18.585 INFO:teuthology.orchestra.run.smithi007.stdout:(25/174): python3-rgw-19.0.0-1855.g21b3bd67.el9 441 kB/s | 99 kB 00:00 2024-03-23T13:07:18.588 INFO:teuthology.orchestra.run.smithi130.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 838 kB 00:00 2024-03-23T13:07:18.610 INFO:teuthology.orchestra.run.smithi007.stdout:(26/174): rbd-fuse-19.0.0-1855.g21b3bd67.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-23T13:07:18.671 INFO:teuthology.orchestra.run.smithi130.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 6.5 MB/s | 548 kB 00:00 2024-03-23T13:07:18.763 INFO:teuthology.orchestra.run.smithi130.stdout:(115/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-03-23T13:07:18.777 INFO:teuthology.orchestra.run.smithi007.stdout:(27/174): ceph-osd-19.0.0-1855.g21b3bd67.el9.x8 3.3 MB/s | 17 MB 00:05 2024-03-23T13:07:18.788 INFO:teuthology.orchestra.run.smithi130.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-03-23T13:07:18.803 INFO:teuthology.orchestra.run.smithi007.stdout:(28/174): rbd-nbd-19.0.0-1855.g21b3bd67.el9.x86 888 kB/s | 170 kB 00:00 2024-03-23T13:07:18.819 INFO:teuthology.orchestra.run.smithi007.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1855.g 759 kB/s | 31 kB 00:00 2024-03-23T13:07:18.830 INFO:teuthology.orchestra.run.smithi130.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-03-23T13:07:18.853 INFO:teuthology.orchestra.run.smithi007.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1855.g21b3bd6 2.7 MB/s | 138 kB 00:00 2024-03-23T13:07:18.855 INFO:teuthology.orchestra.run.smithi130.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-03-23T13:07:18.899 INFO:teuthology.orchestra.run.smithi130.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-23T13:07:18.920 INFO:teuthology.orchestra.run.smithi007.stdout:(31/174): rbd-mirror-19.0.0-1855.g21b3bd67.el9. 9.2 MB/s | 3.1 MB 00:00 2024-03-23T13:07:18.924 INFO:teuthology.orchestra.run.smithi130.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 5.1 MB/s | 358 kB 00:00 2024-03-23T13:07:18.966 INFO:teuthology.orchestra.run.smithi130.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-03-23T13:07:18.987 INFO:teuthology.orchestra.run.smithi007.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1855.g21 3.7 MB/s | 248 kB 00:00 2024-03-23T13:07:18.991 INFO:teuthology.orchestra.run.smithi130.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 526 kB/s | 35 kB 00:00 2024-03-23T13:07:19.033 INFO:teuthology.orchestra.run.smithi130.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-03-23T13:07:19.046 INFO:teuthology.orchestra.run.smithi007.stdout:(33/174): ceph-mgr-rook-19.0.0-1855.g21b3bd67.e 846 kB/s | 49 kB 00:00 2024-03-23T13:07:19.104 INFO:teuthology.orchestra.run.smithi007.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1855.g2 263 kB/s | 15 kB 00:00 2024-03-23T13:07:19.125 INFO:teuthology.orchestra.run.smithi130.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-03-23T13:07:19.151 INFO:teuthology.orchestra.run.smithi130.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-03-23T13:07:19.171 INFO:teuthology.orchestra.run.smithi007.stdout:(35/174): ceph-mgr-dashboard-19.0.0-1855.g21b3b 5.1 MB/s | 1.8 MB 00:00 2024-03-23T13:07:19.193 INFO:teuthology.orchestra.run.smithi130.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-03-23T13:07:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:(36/174): ceph-volume-19.0.0-1855.g21b3bd67.el9 2.8 MB/s | 264 kB 00:00 2024-03-23T13:07:19.218 INFO:teuthology.orchestra.run.smithi130.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-03-23T13:07:19.255 INFO:teuthology.orchestra.run.smithi007.stdout:(37/174): cephfs-top-19.0.0-1855.g21b3bd67.el9. 313 kB/s | 18 kB 00:00 2024-03-23T13:07:19.260 INFO:teuthology.orchestra.run.smithi130.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-03-23T13:07:19.285 INFO:teuthology.orchestra.run.smithi130.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-03-23T13:07:19.322 INFO:teuthology.orchestra.run.smithi007.stdout:(38/174): cephadm-19.0.0-1855.g21b3bd67.el9.noa 4.9 MB/s | 748 kB 00:00 2024-03-23T13:07:19.327 INFO:teuthology.orchestra.run.smithi130.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-03-23T13:07:19.361 INFO:teuthology.orchestra.run.smithi130.stdout:(131/174): protobuf-compiler-3.14.0-13.el9.x86_ 482 kB/s | 863 kB 00:01 2024-03-23T13:07:19.394 INFO:teuthology.orchestra.run.smithi130.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-03-23T13:07:19.420 INFO:teuthology.orchestra.run.smithi130.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-03-23T13:07:19.431 INFO:teuthology.orchestra.run.smithi007.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 8.4 MB/s | 920 kB 00:00 2024-03-23T13:07:19.456 INFO:teuthology.orchestra.run.smithi007.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-03-23T13:07:19.470 INFO:teuthology.orchestra.run.smithi130.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 769 kB/s | 58 kB 00:00 2024-03-23T13:07:19.481 INFO:teuthology.orchestra.run.smithi007.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-23T13:07:19.506 INFO:teuthology.orchestra.run.smithi007.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-23T13:07:19.520 INFO:teuthology.orchestra.run.smithi130.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-03-23T13:07:19.532 INFO:teuthology.orchestra.run.smithi007.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2024-03-23T13:07:19.545 INFO:teuthology.orchestra.run.smithi130.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-03-23T13:07:19.549 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T13:07:19.550 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T13:07:19.562 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T13:07:19.563 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T13:07:19.564 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T13:07:19.565 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T13:07:19.566 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T13:07:19.569 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T13:07:19.570 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T13:07:19.587 INFO:teuthology.orchestra.run.smithi130.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-23T13:07:19.590 INFO:teuthology.orchestra.run.smithi007.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 36 MB/s | 3.0 MB 00:00 2024-03-23T13:07:19.612 INFO:teuthology.orchestra.run.smithi130.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-03-23T13:07:19.616 INFO:teuthology.orchestra.run.smithi007.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 213 kB/s | 17 kB 00:00 2024-03-23T13:07:19.641 INFO:teuthology.orchestra.run.smithi007.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2024-03-23T13:07:19.649 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-23T13:07:19.655 INFO:teuthology.orchestra.run.smithi130.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 6.6 k 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.3 M 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 888 k 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 145 k 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 1.5 M 2024-03-23T13:07:19.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 138 k 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 1.8 M 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 7.4 M 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 49 k 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 9.8 M 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 47 M 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 264 k 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 748 k 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 219 k 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 27 k 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 18 k 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T13:07:19.663 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 32 k 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 744 k 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 127 k 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 161 k 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 320 k 2024-03-23T13:07:19.664 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 298 k 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 99 k 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 85 k 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 170 k 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T13:07:19.665 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.4 M 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T13:07:19.666 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 21 M 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 31 k 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 2.4 M 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 248 k 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 4.7 M 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 17 M 2024-03-23T13:07:19.667 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 15 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 25 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 163 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T13:07:19.668 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 499 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.1 M 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-23T13:07:19.669 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 45 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 131 k 2024-03-23T13:07:19.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T13:07:19.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T13:07:19.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T13:07:19.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T13:07:19.674 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T13:07:19.675 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T13:07:19.675 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T13:07:19.675 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-03-23T13:07:19.675 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-23T13:07:19.675 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:07:19.675 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-23T13:07:19.675 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2024-03-23T13:07:19.675 INFO:teuthology.orchestra.run.smithi155.stdout:Install 150 Packages 2024-03-23T13:07:19.675 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 24 Packages 2024-03-23T13:07:19.675 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:07:19.676 INFO:teuthology.orchestra.run.smithi007.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-23T13:07:19.676 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 224 M 2024-03-23T13:07:19.676 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-23T13:07:19.680 INFO:teuthology.orchestra.run.smithi130.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-03-23T13:07:19.691 INFO:teuthology.orchestra.run.smithi007.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-23T13:07:19.706 INFO:teuthology.orchestra.run.smithi130.stdout:(141/174): python3-more-itertools-8.12.0-2.el9. 229 kB/s | 79 kB 00:00 2024-03-23T13:07:19.725 INFO:teuthology.orchestra.run.smithi007.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-23T13:07:19.731 INFO:teuthology.orchestra.run.smithi130.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-03-23T13:07:19.751 INFO:teuthology.orchestra.run.smithi007.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-03-23T13:07:19.756 INFO:teuthology.orchestra.run.smithi130.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-03-23T13:07:19.789 INFO:teuthology.orchestra.run.smithi130.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 609 kB/s | 51 kB 00:00 2024-03-23T13:07:19.815 INFO:teuthology.orchestra.run.smithi130.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-23T13:07:19.840 INFO:teuthology.orchestra.run.smithi130.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-23T13:07:19.882 INFO:teuthology.orchestra.run.smithi130.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-03-23T13:07:19.901 INFO:teuthology.orchestra.run.smithi007.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 266 kB/s | 46 kB 00:00 2024-03-23T13:07:19.909 INFO:teuthology.orchestra.run.smithi130.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-23T13:07:19.926 INFO:teuthology.orchestra.run.smithi007.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 282 kB 00:00 2024-03-23T13:07:19.951 INFO:teuthology.orchestra.run.smithi007.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-23T13:07:19.985 INFO:teuthology.orchestra.run.smithi007.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-03-23T13:07:19.999 INFO:teuthology.orchestra.run.smithi130.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-03-23T13:07:20.002 INFO:teuthology.orchestra.run.smithi007.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-03-23T13:07:20.033 INFO:teuthology.orchestra.run.smithi130.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-23T13:07:20.102 INFO:teuthology.orchestra.run.smithi007.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-03-23T13:07:20.136 INFO:teuthology.orchestra.run.smithi007.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-03-23T13:07:20.177 INFO:teuthology.orchestra.run.smithi007.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-23T13:07:20.191 INFO:teuthology.orchestra.run.smithi130.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.0 MB/s | 321 kB 00:00 2024-03-23T13:07:20.203 INFO:teuthology.orchestra.run.smithi007.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2024-03-23T13:07:20.295 INFO:teuthology.orchestra.run.smithi007.stdout:(60/174): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-03-23T13:07:20.378 INFO:teuthology.orchestra.run.smithi007.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 28 MB/s | 5.7 MB 00:00 2024-03-23T13:07:20.412 INFO:teuthology.orchestra.run.smithi007.stdout:(62/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-03-23T13:07:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 557 kB/s | 78 kB 00:00 2024-03-23T13:07:20.507 INFO:teuthology.orchestra.run.smithi155.stdout:(1/174): ceph-19.0.0-1855.g21b3bd67.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-03-23T13:07:20.546 INFO:teuthology.orchestra.run.smithi007.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 5.8 MB 00:00 2024-03-23T13:07:20.571 INFO:teuthology.orchestra.run.smithi007.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 187 kB/s | 24 kB 00:00 2024-03-23T13:07:20.596 INFO:teuthology.orchestra.run.smithi007.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-03-23T13:07:20.621 INFO:teuthology.orchestra.run.smithi007.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-23T13:07:20.647 INFO:teuthology.orchestra.run.smithi007.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2024-03-23T13:07:20.658 INFO:teuthology.orchestra.run.smithi130.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 4.9 MB/s | 2.3 MB 00:00 2024-03-23T13:07:20.722 INFO:teuthology.orchestra.run.smithi007.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-23T13:07:20.747 INFO:teuthology.orchestra.run.smithi007.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-03-23T13:07:20.773 INFO:teuthology.orchestra.run.smithi007.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-23T13:07:20.798 INFO:teuthology.orchestra.run.smithi007.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 922 kB/s | 69 kB 00:00 2024-03-23T13:07:20.815 INFO:teuthology.orchestra.run.smithi155.stdout:(2/174): ceph-base-19.0.0-1855.g21b3bd67.el9.x8 9.8 MB/s | 5.3 MB 00:00 2024-03-23T13:07:20.823 INFO:teuthology.orchestra.run.smithi007.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 826 kB/s | 41 kB 00:00 2024-03-23T13:07:20.848 INFO:teuthology.orchestra.run.smithi007.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-23T13:07:20.866 INFO:teuthology.orchestra.run.smithi155.stdout:(3/174): ceph-immutable-object-cache-19.0.0-185 2.8 MB/s | 145 kB 00:00 2024-03-23T13:07:20.873 INFO:teuthology.orchestra.run.smithi007.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-23T13:07:20.899 INFO:teuthology.orchestra.run.smithi007.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-23T13:07:20.916 INFO:teuthology.orchestra.run.smithi007.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 710 kB/s | 47 kB 00:00 2024-03-23T13:07:20.925 INFO:teuthology.orchestra.run.smithi130.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 6.9 MB/s | 1.8 MB 00:00 2024-03-23T13:07:20.932 INFO:teuthology.orchestra.run.smithi007.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 735 kB/s | 43 kB 00:00 2024-03-23T13:07:20.949 INFO:teuthology.orchestra.run.smithi007.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-23T13:07:20.975 INFO:teuthology.orchestra.run.smithi130.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 844 kB/s | 42 kB 00:00 2024-03-23T13:07:21.034 INFO:teuthology.orchestra.run.smithi130.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-23T13:07:21.041 INFO:teuthology.orchestra.run.smithi007.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 41 kB 00:00 2024-03-23T13:07:21.049 INFO:teuthology.orchestra.run.smithi155.stdout:(4/174): ceph-mds-19.0.0-1855.g21b3bd67.el9.x86 13 MB/s | 2.4 MB 00:00 2024-03-23T13:07:21.076 INFO:teuthology.orchestra.run.smithi130.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-23T13:07:21.117 INFO:teuthology.orchestra.run.smithi007.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.8 MB/s | 300 kB 00:00 2024-03-23T13:07:21.118 INFO:teuthology.orchestra.run.smithi130.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 876 kB/s | 36 kB 00:00 2024-03-23T13:07:21.166 INFO:teuthology.orchestra.run.smithi155.stdout:(5/174): ceph-mgr-19.0.0-1855.g21b3bd67.el9.x86 13 MB/s | 1.5 MB 00:00 2024-03-23T13:07:21.193 INFO:teuthology.orchestra.run.smithi130.stdout:(158/174): librbd1-19.0.0-1855.g21b3bd67.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-03-23T13:07:21.199 INFO:teuthology.orchestra.run.smithi155.stdout:(6/174): ceph-fuse-19.0.0-1855.g21b3bd67.el9.x8 1.3 MB/s | 888 kB 00:00 2024-03-23T13:07:21.218 INFO:teuthology.orchestra.run.smithi130.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.9 MB/s | 192 kB 00:00 2024-03-23T13:07:21.292 INFO:teuthology.orchestra.run.smithi007.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-03-23T13:07:21.294 INFO:teuthology.orchestra.run.smithi130.stdout:(160/174): librados2-19.0.0-1855.g21b3bd67.el9. 2.4 MB/s | 3.4 MB 00:01 2024-03-23T13:07:21.308 INFO:teuthology.orchestra.run.smithi007.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 213 kB/s | 56 kB 00:00 2024-03-23T13:07:21.319 INFO:teuthology.orchestra.run.smithi130.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 229 kB/s | 23 kB 00:00 2024-03-23T13:07:21.325 INFO:teuthology.orchestra.run.smithi007.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 302 kB/s | 63 kB 00:00 2024-03-23T13:07:21.351 INFO:teuthology.orchestra.run.smithi007.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.9 MB/s | 79 kB 00:00 2024-03-23T13:07:21.368 INFO:teuthology.orchestra.run.smithi007.stdout:(86/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 753 kB/s | 44 kB 00:00 2024-03-23T13:07:21.378 INFO:teuthology.orchestra.run.smithi130.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-23T13:07:21.393 INFO:teuthology.orchestra.run.smithi007.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-23T13:07:21.428 INFO:teuthology.orchestra.run.smithi130.stdout:(163/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.1 MB/s | 256 kB 00:00 2024-03-23T13:07:21.433 INFO:teuthology.orchestra.run.smithi155.stdout:(7/174): ceph-mon-19.0.0-1855.g21b3bd67.el9.x86 18 MB/s | 4.7 MB 00:00 2024-03-23T13:07:21.435 INFO:teuthology.orchestra.run.smithi007.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-23T13:07:21.452 INFO:teuthology.orchestra.run.smithi007.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-23T13:07:21.453 INFO:teuthology.orchestra.run.smithi130.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 681 kB/s | 108 kB 00:00 2024-03-23T13:07:21.477 INFO:teuthology.orchestra.run.smithi007.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-23T13:07:21.478 INFO:teuthology.orchestra.run.smithi130.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-23T13:07:21.502 INFO:teuthology.orchestra.run.smithi007.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2024-03-23T13:07:21.503 INFO:teuthology.orchestra.run.smithi130.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 848 kB/s | 64 kB 00:00 2024-03-23T13:07:21.529 INFO:teuthology.orchestra.run.smithi130.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 380 kB/s | 29 kB 00:00 2024-03-23T13:07:21.654 INFO:teuthology.orchestra.run.smithi130.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 6.9 MB/s | 1.2 MB 00:00 2024-03-23T13:07:21.711 INFO:teuthology.orchestra.run.smithi007.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 938 kB/s | 195 kB 00:00 2024-03-23T13:07:21.736 INFO:teuthology.orchestra.run.smithi007.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 724 kB/s | 187 kB 00:00 2024-03-23T13:07:21.746 INFO:teuthology.orchestra.run.smithi130.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 5.0 MB/s | 465 kB 00:00 2024-03-23T13:07:21.770 INFO:teuthology.orchestra.run.smithi007.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-23T13:07:21.822 INFO:teuthology.orchestra.run.smithi130.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.8 MB/s | 2.2 MB 00:00 2024-03-23T13:07:21.847 INFO:teuthology.orchestra.run.smithi130.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 310 kB/s | 31 kB 00:00 2024-03-23T13:07:21.878 INFO:teuthology.orchestra.run.smithi007.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-03-23T13:07:21.914 INFO:teuthology.orchestra.run.smithi130.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.9 MB/s | 2.3 MB 00:00 2024-03-23T13:07:21.967 INFO:teuthology.orchestra.run.smithi155.stdout:(8/174): ceph-radosgw-19.0.0-1855.g21b3bd67.el9 18 MB/s | 9.8 MB 00:00 2024-03-23T13:07:21.989 INFO:teuthology.orchestra.run.smithi130.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 524 kB/s | 74 kB 00:00 2024-03-23T13:07:22.004 INFO:teuthology.orchestra.run.smithi007.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 80 kB/s | 21 kB 00:00 2024-03-23T13:07:22.017 INFO:teuthology.orchestra.run.smithi155.stdout:(9/174): ceph-selinux-19.0.0-1855.g21b3bd67.el9 506 kB/s | 25 kB 00:00 2024-03-23T13:07:22.022 INFO:teuthology.orchestra.run.smithi130.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.4 MB/s | 1.1 MB 00:00 2024-03-23T13:07:22.029 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-03-23T13:07:22.030 INFO:teuthology.orchestra.run.smithi130.stdout:Total 16 MB/s | 224 MB 00:13 2024-03-23T13:07:22.062 INFO:teuthology.orchestra.run.smithi007.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 87 kB/s | 25 kB 00:00 2024-03-23T13:07:22.429 INFO:teuthology.orchestra.run.smithi007.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 577 kB/s | 245 kB 00:00 2024-03-23T13:07:22.480 INFO:teuthology.orchestra.run.smithi007.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 594 kB/s | 248 kB 00:00 2024-03-23T13:07:22.510 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-03-23T13:07:22.588 INFO:teuthology.orchestra.run.smithi007.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 322 kB/s | 35 kB 00:00 2024-03-23T13:07:22.614 INFO:teuthology.orchestra.run.smithi007.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.1 MB/s | 216 kB 00:00 2024-03-23T13:07:22.697 INFO:teuthology.orchestra.run.smithi007.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 410 kB/s | 44 kB 00:00 2024-03-23T13:07:22.722 INFO:teuthology.orchestra.run.smithi007.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 863 kB/s | 94 kB 00:00 2024-03-23T13:07:22.781 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-03-23T13:07:22.782 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-03-23T13:07:22.789 INFO:teuthology.orchestra.run.smithi007.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-03-23T13:07:22.973 INFO:teuthology.orchestra.run.smithi007.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.9 MB/s | 548 kB 00:00 2024-03-23T13:07:23.048 INFO:teuthology.orchestra.run.smithi007.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-03-23T13:07:23.149 INFO:teuthology.orchestra.run.smithi007.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-03-23T13:07:23.174 INFO:teuthology.orchestra.run.smithi007.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-03-23T13:07:23.299 INFO:teuthology.orchestra.run.smithi007.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-03-23T13:07:23.433 INFO:teuthology.orchestra.run.smithi007.stdout:(110/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.7 MB/s | 4.1 MB 00:01 2024-03-23T13:07:23.458 INFO:teuthology.orchestra.run.smithi007.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 306 kB/s | 49 kB 00:00 2024-03-23T13:07:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 384 kB/s | 67 kB 00:00 2024-03-23T13:07:23.633 INFO:teuthology.orchestra.run.smithi007.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-23T13:07:23.809 INFO:teuthology.orchestra.run.smithi007.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-03-23T13:07:23.909 INFO:teuthology.orchestra.run.smithi007.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-03-23T13:07:23.942 INFO:teuthology.orchestra.run.smithi007.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-03-23T13:07:23.984 INFO:teuthology.orchestra.run.smithi007.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 248 kB/s | 43 kB 00:00 2024-03-23T13:07:24.085 INFO:teuthology.orchestra.run.smithi007.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 184 kB/s | 32 kB 00:00 2024-03-23T13:07:24.110 INFO:teuthology.orchestra.run.smithi007.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 105 kB/s | 15 kB 00:00 2024-03-23T13:07:24.210 INFO:teuthology.orchestra.run.smithi007.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 857 kB/s | 172 kB 00:00 2024-03-23T13:07:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-03-23T13:07:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.6 MB/s | 303 kB 00:00 2024-03-23T13:07:24.361 INFO:teuthology.orchestra.run.smithi007.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 235 kB/s | 35 kB 00:00 2024-03-23T13:07:24.367 INFO:teuthology.orchestra.run.smithi155.stdout:(10/174): ceph-common-19.0.0-1855.g21b3bd67.el9 5.0 MB/s | 21 MB 00:04 2024-03-23T13:07:24.467 INFO:teuthology.orchestra.run.smithi155.stdout:(11/174): cephfs-mirror-19.0.0-1855.g21b3bd67.e 2.1 MB/s | 219 kB 00:00 2024-03-23T13:07:24.469 INFO:teuthology.orchestra.run.smithi007.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.1 MB/s | 203 kB 00:00 2024-03-23T13:07:24.474 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-03-23T13:07:24.475 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-03-23T13:07:24.545 INFO:teuthology.orchestra.run.smithi007.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 913 kB/s | 144 kB 00:00 2024-03-23T13:07:24.567 INFO:teuthology.orchestra.run.smithi155.stdout:(12/174): cephfs-shell-19.0.0-1855.g21b3bd67.el 271 kB/s | 27 kB 00:00 2024-03-23T13:07:24.595 INFO:teuthology.orchestra.run.smithi007.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-03-23T13:07:24.653 INFO:teuthology.orchestra.run.smithi007.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-03-23T13:07:24.676 INFO:teuthology.orchestra.run.smithi155.stdout:(13/174): libcephfs-devel-19.0.0-1855.g21b3bd67 293 kB/s | 32 kB 00:00 2024-03-23T13:07:24.745 INFO:teuthology.orchestra.run.smithi007.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.1 MB/s | 2.0 MB 00:00 2024-03-23T13:07:24.770 INFO:teuthology.orchestra.run.smithi007.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 133 kB/s | 23 kB 00:00 2024-03-23T13:07:24.829 INFO:teuthology.orchestra.run.smithi007.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 111 kB/s | 19 kB 00:00 2024-03-23T13:07:24.910 INFO:teuthology.orchestra.run.smithi155.stdout:(14/174): ceph-osd-19.0.0-1855.g21b3bd67.el9.x8 4.5 MB/s | 17 MB 00:03 2024-03-23T13:07:24.913 INFO:teuthology.orchestra.run.smithi007.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 118 kB/s | 20 kB 00:00 2024-03-23T13:07:24.944 INFO:teuthology.orchestra.run.smithi155.stdout:(15/174): libcephfs2-19.0.0-1855.g21b3bd67.el9. 2.7 MB/s | 744 kB 00:00 2024-03-23T13:07:24.980 INFO:teuthology.orchestra.run.smithi007.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 371 kB/s | 46 kB 00:00 2024-03-23T13:07:25.047 INFO:teuthology.orchestra.run.smithi007.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 4.1 MB/s | 1.0 MB 00:00 2024-03-23T13:07:25.072 INFO:teuthology.orchestra.run.smithi007.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 499 kB/s | 79 kB 00:00 2024-03-23T13:07:25.123 INFO:teuthology.orchestra.run.smithi007.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 406 kB/s | 58 kB 00:00 2024-03-23T13:07:25.231 INFO:teuthology.orchestra.run.smithi007.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 122 kB/s | 16 kB 00:00 2024-03-23T13:07:25.256 INFO:teuthology.orchestra.run.smithi007.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-03-23T13:07:25.307 INFO:teuthology.orchestra.run.smithi007.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 568 kB/s | 90 kB 00:00 2024-03-23T13:07:25.390 INFO:teuthology.orchestra.run.smithi007.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 178 kB/s | 28 kB 00:00 2024-03-23T13:07:25.394 INFO:teuthology.orchestra.run.smithi155.stdout:(16/174): ceph-test-19.0.0-1855.g21b3bd67.el9.x 14 MB/s | 47 MB 00:03 2024-03-23T13:07:25.411 INFO:teuthology.orchestra.run.smithi155.stdout:(17/174): librados-devel-19.0.0-1855.g21b3bd67. 271 kB/s | 127 kB 00:00 2024-03-23T13:07:25.436 INFO:teuthology.orchestra.run.smithi155.stdout:(18/174): libcephsqlite-19.0.0-1855.g21b3bd67.e 310 kB/s | 163 kB 00:00 2024-03-23T13:07:25.449 INFO:teuthology.orchestra.run.smithi007.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 183 kB/s | 31 kB 00:00 2024-03-23T13:07:25.469 INFO:teuthology.orchestra.run.smithi155.stdout:(19/174): libradosstriper1-19.0.0-1855.g21b3bd6 6.5 MB/s | 499 kB 00:00 2024-03-23T13:07:25.486 INFO:teuthology.orchestra.run.smithi155.stdout:(20/174): python3-ceph-argparse-19.0.0-1855.g21 902 kB/s | 45 kB 00:00 2024-03-23T13:07:25.507 INFO:teuthology.orchestra.run.smithi007.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-03-23T13:07:25.528 INFO:teuthology.orchestra.run.smithi155.stdout:(21/174): python3-ceph-common-19.0.0-1855.g21b3 2.2 MB/s | 131 kB 00:00 2024-03-23T13:07:25.533 INFO:teuthology.orchestra.run.smithi007.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 419 kB/s | 59 kB 00:00 2024-03-23T13:07:25.553 INFO:teuthology.orchestra.run.smithi155.stdout:(22/174): python3-cephfs-19.0.0-1855.g21b3bd67. 2.4 MB/s | 161 kB 00:00 2024-03-23T13:07:25.591 INFO:teuthology.orchestra.run.smithi007.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 254 kB/s | 36 kB 00:00 2024-03-23T13:07:25.595 INFO:teuthology.orchestra.run.smithi155.stdout:(23/174): python3-rados-19.0.0-1855.g21b3bd67.e 4.7 MB/s | 320 kB 00:00 2024-03-23T13:07:25.621 INFO:teuthology.orchestra.run.smithi155.stdout:(24/174): python3-rbd-19.0.0-1855.g21b3bd67.el9 4.4 MB/s | 298 kB 00:00 2024-03-23T13:07:25.646 INFO:teuthology.orchestra.run.smithi155.stdout:(25/174): python3-rgw-19.0.0-1855.g21b3bd67.el9 1.9 MB/s | 99 kB 00:00 2024-03-23T13:07:25.683 INFO:teuthology.orchestra.run.smithi007.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 292 kB/s | 51 kB 00:00 2024-03-23T13:07:25.688 INFO:teuthology.orchestra.run.smithi155.stdout:(26/174): rbd-fuse-19.0.0-1855.g21b3bd67.el9.x8 1.2 MB/s | 85 kB 00:00 2024-03-23T13:07:25.758 INFO:teuthology.orchestra.run.smithi007.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 633 kB/s | 90 kB 00:00 2024-03-23T13:07:25.788 INFO:teuthology.orchestra.run.smithi155.stdout:(27/174): rbd-nbd-19.0.0-1855.g21b3bd67.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-23T13:07:25.850 INFO:teuthology.orchestra.run.smithi007.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 784 kB/s | 230 kB 00:00 2024-03-23T13:07:25.872 INFO:teuthology.orchestra.run.smithi155.stdout:(28/174): rbd-mirror-19.0.0-1855.g21b3bd67.el9. 14 MB/s | 3.1 MB 00:00 2024-03-23T13:07:25.889 INFO:teuthology.orchestra.run.smithi155.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1855.g 313 kB/s | 31 kB 00:00 2024-03-23T13:07:25.909 INFO:teuthology.orchestra.run.smithi007.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 133 kB/s | 20 kB 00:00 2024-03-23T13:07:25.914 INFO:teuthology.orchestra.run.smithi155.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1855.g21b3bd6 3.2 MB/s | 138 kB 00:00 2024-03-23T13:07:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2024-03-23T13:07:25.981 INFO:teuthology.orchestra.run.smithi155.stdout:(31/174): librgw2-19.0.0-1855.g21b3bd67.el9.x86 9.0 MB/s | 5.1 MB 00:00 2024-03-23T13:07:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-03-23T13:07:26.090 INFO:teuthology.orchestra.run.smithi155.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1855.g21 2.2 MB/s | 248 kB 00:00 2024-03-23T13:07:26.124 INFO:teuthology.orchestra.run.smithi155.stdout:(33/174): ceph-mgr-dashboard-19.0.0-1855.g21b3b 7.6 MB/s | 1.8 MB 00:00 2024-03-23T13:07:26.165 INFO:teuthology.orchestra.run.smithi155.stdout:(34/174): ceph-mgr-rook-19.0.0-1855.g21b3bd67.e 658 kB/s | 49 kB 00:00 2024-03-23T13:07:26.176 INFO:teuthology.orchestra.run.smithi007.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 5.9 MB/s | 1.6 MB 00:00 2024-03-23T13:07:26.199 INFO:teuthology.orchestra.run.smithi155.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1855.g2 204 kB/s | 15 kB 00:00 2024-03-23T13:07:26.218 INFO:teuthology.orchestra.run.smithi007.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-23T13:07:26.257 INFO:teuthology.orchestra.run.smithi155.stdout:(36/174): ceph-volume-19.0.0-1855.g21b3bd67.el9 2.8 MB/s | 264 kB 00:00 2024-03-23T13:07:26.391 INFO:teuthology.orchestra.run.smithi155.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-03-23T13:07:26.417 INFO:teuthology.orchestra.run.smithi155.stdout:(38/174): cephfs-top-19.0.0-1855.g21b3bd67.el9. 115 kB/s | 18 kB 00:00 2024-03-23T13:07:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:(152/174): librbd1-19.0.0-1855.g21b3bd67.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-03-23T13:07:26.567 INFO:teuthology.orchestra.run.smithi155.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 6.0 MB/s | 920 kB 00:00 2024-03-23T13:07:26.592 INFO:teuthology.orchestra.run.smithi155.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-03-23T13:07:26.617 INFO:teuthology.orchestra.run.smithi155.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-23T13:07:26.642 INFO:teuthology.orchestra.run.smithi155.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-23T13:07:26.668 INFO:teuthology.orchestra.run.smithi155.stdout:(43/174): cephadm-19.0.0-1855.g21b3bd67.el9.noa 1.6 MB/s | 748 kB 00:00 2024-03-23T13:07:26.693 INFO:teuthology.orchestra.run.smithi155.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 422 kB/s | 32 kB 00:00 2024-03-23T13:07:26.710 INFO:teuthology.orchestra.run.smithi007.stdout:(153/174): librados2-19.0.0-1855.g21b3bd67.el9. 4.5 MB/s | 3.4 MB 00:00 2024-03-23T13:07:26.743 INFO:teuthology.orchestra.run.smithi155.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 209 kB/s | 10 kB 00:00 2024-03-23T13:07:26.802 INFO:teuthology.orchestra.run.smithi155.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-03-23T13:07:26.827 INFO:teuthology.orchestra.run.smithi155.stdout:(47/174): flexiblas-openblas-openmp-3.0.4-8.el9 130 kB/s | 17 kB 00:00 2024-03-23T13:07:26.835 INFO:teuthology.orchestra.run.smithi007.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 3.7 MB/s | 2.3 MB 00:00 2024-03-23T13:07:26.854 INFO:teuthology.orchestra.run.smithi155.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-03-23T13:07:26.861 INFO:teuthology.orchestra.run.smithi007.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 280 kB/s | 42 kB 00:00 2024-03-23T13:07:26.877 INFO:teuthology.orchestra.run.smithi155.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 888 kB/s | 44 kB 00:00 2024-03-23T13:07:26.894 INFO:teuthology.orchestra.run.smithi007.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.1 MB/s | 248 kB 00:00 2024-03-23T13:07:26.911 INFO:teuthology.orchestra.run.smithi155.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-03-23T13:07:26.936 INFO:teuthology.orchestra.run.smithi155.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.9 MB/s | 578 kB 00:00 2024-03-23T13:07:26.953 INFO:teuthology.orchestra.run.smithi155.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-23T13:07:26.953 INFO:teuthology.orchestra.run.smithi007.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 626 kB/s | 36 kB 00:00 2024-03-23T13:07:26.978 INFO:teuthology.orchestra.run.smithi155.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-23T13:07:26.978 INFO:teuthology.orchestra.run.smithi007.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 609 kB/s | 71 kB 00:00 2024-03-23T13:07:27.003 INFO:teuthology.orchestra.run.smithi155.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2024-03-23T13:07:27.012 INFO:teuthology.orchestra.run.smithi007.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-23T13:07:27.020 INFO:teuthology.orchestra.run.smithi155.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 828 kB/s | 34 kB 00:00 2024-03-23T13:07:27.046 INFO:teuthology.orchestra.run.smithi155.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-03-23T13:07:27.071 INFO:teuthology.orchestra.run.smithi007.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 395 kB/s | 23 kB 00:00 2024-03-23T13:07:27.096 INFO:teuthology.orchestra.run.smithi007.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-03-23T13:07:27.138 INFO:teuthology.orchestra.run.smithi007.stdout:(162/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.0 MB/s | 1.8 MB 00:00 2024-03-23T13:07:27.138 INFO:teuthology.orchestra.run.smithi155.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-03-23T13:07:27.163 INFO:teuthology.orchestra.run.smithi007.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-03-23T13:07:27.174 INFO:teuthology.orchestra.run.smithi155.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-03-23T13:07:27.188 INFO:teuthology.orchestra.run.smithi007.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 155 kB 00:00 2024-03-23T13:07:27.199 INFO:teuthology.orchestra.run.smithi155.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 5.3 MB/s | 811 kB 00:00 2024-03-23T13:07:27.214 INFO:teuthology.orchestra.run.smithi007.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.7 MB/s | 136 kB 00:00 2024-03-23T13:07:27.224 INFO:teuthology.orchestra.run.smithi155.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.6 MB/s | 134 kB 00:00 2024-03-23T13:07:27.239 INFO:teuthology.orchestra.run.smithi007.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 844 kB/s | 64 kB 00:00 2024-03-23T13:07:27.256 INFO:teuthology.orchestra.run.smithi007.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 426 kB/s | 29 kB 00:00 2024-03-23T13:07:27.266 INFO:teuthology.orchestra.run.smithi155.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-23T13:07:27.358 INFO:teuthology.orchestra.run.smithi155.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-23T13:07:27.383 INFO:teuthology.orchestra.run.smithi155.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 488 kB/s | 78 kB 00:00 2024-03-23T13:07:27.476 INFO:teuthology.orchestra.run.smithi155.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-03-23T13:07:27.501 INFO:teuthology.orchestra.run.smithi155.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-23T13:07:27.506 INFO:teuthology.orchestra.run.smithi007.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 4.1 MB/s | 1.2 MB 00:00 2024-03-23T13:07:27.534 INFO:teuthology.orchestra.run.smithi155.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 122 kB/s | 18 kB 00:00 2024-03-23T13:07:27.559 INFO:teuthology.orchestra.run.smithi155.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-23T13:07:27.585 INFO:teuthology.orchestra.run.smithi155.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-23T13:07:27.607 INFO:teuthology.orchestra.run.smithi007.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.9 MB/s | 2.2 MB 00:00 2024-03-23T13:07:27.632 INFO:teuthology.orchestra.run.smithi007.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-03-23T13:07:27.660 INFO:teuthology.orchestra.run.smithi155.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-23T13:07:27.665 INFO:teuthology.orchestra.run.smithi007.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 533 kB/s | 31 kB 00:00 2024-03-23T13:07:27.686 INFO:teuthology.orchestra.run.smithi155.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-03-23T13:07:27.710 INFO:teuthology.orchestra.run.smithi155.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-23T13:07:27.736 INFO:teuthology.orchestra.run.smithi155.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-03-23T13:07:27.749 INFO:teuthology.orchestra.run.smithi007.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.6 MB/s | 2.3 MB 00:00 2024-03-23T13:07:27.761 INFO:teuthology.orchestra.run.smithi155.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-03-23T13:07:27.786 INFO:teuthology.orchestra.run.smithi155.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-23T13:07:27.812 INFO:teuthology.orchestra.run.smithi155.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-23T13:07:27.816 INFO:teuthology.orchestra.run.smithi007.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-03-23T13:07:27.837 INFO:teuthology.orchestra.run.smithi155.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 952 kB/s | 47 kB 00:00 2024-03-23T13:07:27.862 INFO:teuthology.orchestra.run.smithi155.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-23T13:07:27.887 INFO:teuthology.orchestra.run.smithi155.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-03-23T13:07:27.958 INFO:teuthology.orchestra.run.smithi007.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 254 kB/s | 74 kB 00:00 2024-03-23T13:07:27.962 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-23T13:07:27.962 INFO:teuthology.orchestra.run.smithi007.stdout:Total 12 MB/s | 224 MB 00:18 2024-03-23T13:07:28.204 INFO:teuthology.orchestra.run.smithi155.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-03-23T13:07:28.230 INFO:teuthology.orchestra.run.smithi155.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 114 kB/s | 39 kB 00:00 2024-03-23T13:07:28.255 INFO:teuthology.orchestra.run.smithi155.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 112 kB/s | 41 kB 00:00 2024-03-23T13:07:28.280 INFO:teuthology.orchestra.run.smithi155.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-03-23T13:07:28.305 INFO:teuthology.orchestra.run.smithi155.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-23T13:07:28.322 INFO:teuthology.orchestra.run.smithi155.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 946 kB/s | 63 kB 00:00 2024-03-23T13:07:28.339 INFO:teuthology.orchestra.run.smithi155.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 753 kB/s | 44 kB 00:00 2024-03-23T13:07:28.364 INFO:teuthology.orchestra.run.smithi155.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-23T13:07:28.390 INFO:teuthology.orchestra.run.smithi155.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-23T13:07:28.423 INFO:teuthology.orchestra.run.smithi155.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-23T13:07:28.439 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-23T13:07:28.440 INFO:teuthology.orchestra.run.smithi155.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-23T13:07:28.465 INFO:teuthology.orchestra.run.smithi155.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 124 kB 00:00 2024-03-23T13:07:28.490 INFO:teuthology.orchestra.run.smithi155.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-23T13:07:28.616 INFO:teuthology.orchestra.run.smithi155.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-23T13:07:28.641 INFO:teuthology.orchestra.run.smithi155.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-23T13:07:28.658 INFO:teuthology.orchestra.run.smithi155.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-03-23T13:07:28.691 INFO:teuthology.orchestra.run.smithi155.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-03-23T13:07:28.735 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-23T13:07:28.736 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-23T13:07:28.838 INFO:teuthology.orchestra.run.smithi155.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 109 kB/s | 21 kB 00:00 2024-03-23T13:07:28.867 INFO:teuthology.orchestra.run.smithi155.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 122 kB/s | 25 kB 00:00 2024-03-23T13:07:29.102 INFO:teuthology.orchestra.run.smithi155.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 936 kB/s | 245 kB 00:00 2024-03-23T13:07:29.126 INFO:teuthology.orchestra.run.smithi155.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 959 kB/s | 248 kB 00:00 2024-03-23T13:07:29.260 INFO:teuthology.orchestra.run.smithi155.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 261 kB/s | 35 kB 00:00 2024-03-23T13:07:29.393 INFO:teuthology.orchestra.run.smithi155.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 741 kB/s | 216 kB 00:00 2024-03-23T13:07:29.418 INFO:teuthology.orchestra.run.smithi155.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 281 kB/s | 44 kB 00:00 2024-03-23T13:07:29.585 INFO:teuthology.orchestra.run.smithi155.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 488 kB/s | 94 kB 00:00 2024-03-23T13:07:29.685 INFO:teuthology.orchestra.run.smithi155.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 84 kB/s | 22 kB 00:00 2024-03-23T13:07:30.019 INFO:teuthology.orchestra.run.smithi155.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-03-23T13:07:30.078 INFO:teuthology.orchestra.run.smithi155.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.0 MB/s | 4.1 MB 00:01 2024-03-23T13:07:30.103 INFO:teuthology.orchestra.run.smithi155.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-03-23T13:07:30.195 INFO:teuthology.orchestra.run.smithi155.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-03-23T13:07:30.262 INFO:teuthology.orchestra.run.smithi155.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-03-23T13:07:30.304 INFO:teuthology.orchestra.run.smithi155.stdout:(110/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-03-23T13:07:30.371 INFO:teuthology.orchestra.run.smithi155.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-03-23T13:07:30.396 INFO:teuthology.orchestra.run.smithi155.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 363 kB/s | 49 kB 00:00 2024-03-23T13:07:30.438 INFO:teuthology.orchestra.run.smithi155.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-23T13:07:30.513 INFO:teuthology.orchestra.run.smithi155.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-03-23T13:07:30.539 INFO:teuthology.orchestra.run.smithi155.stdout:(115/174): libunwind-1.6.2-1.el9.x86_64.rpm 287 kB/s | 67 kB 00:00 2024-03-23T13:07:30.564 INFO:teuthology.orchestra.run.smithi155.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-03-23T13:07:30.606 INFO:teuthology.orchestra.run.smithi155.stdout:(117/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-03-23T13:07:30.631 INFO:teuthology.orchestra.run.smithi155.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-03-23T13:07:30.656 INFO:teuthology.orchestra.run.smithi155.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-03-23T13:07:30.682 INFO:teuthology.orchestra.run.smithi155.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-23T13:07:30.724 INFO:teuthology.orchestra.run.smithi155.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-03-23T13:07:30.741 INFO:teuthology.orchestra.run.smithi155.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-03-23T13:07:30.791 INFO:teuthology.orchestra.run.smithi155.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-03-23T13:07:30.813 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-23T13:07:30.813 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-23T13:07:30.816 INFO:teuthology.orchestra.run.smithi155.stdout:(124/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-03-23T13:07:30.867 INFO:teuthology.orchestra.run.smithi155.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-03-23T13:07:30.883 INFO:teuthology.orchestra.run.smithi155.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-03-23T13:07:30.917 INFO:teuthology.orchestra.run.smithi155.stdout:(127/174): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-03-23T13:07:30.942 INFO:teuthology.orchestra.run.smithi155.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-03-23T13:07:30.967 INFO:teuthology.orchestra.run.smithi155.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-03-23T13:07:30.993 INFO:teuthology.orchestra.run.smithi155.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-03-23T13:07:31.018 INFO:teuthology.orchestra.run.smithi155.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-03-23T13:07:31.043 INFO:teuthology.orchestra.run.smithi155.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 925 kB/s | 46 kB 00:00 2024-03-23T13:07:31.077 INFO:teuthology.orchestra.run.smithi155.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-23T13:07:31.102 INFO:teuthology.orchestra.run.smithi155.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 984 kB/s | 58 kB 00:00 2024-03-23T13:07:31.136 INFO:teuthology.orchestra.run.smithi155.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-03-23T13:07:31.161 INFO:teuthology.orchestra.run.smithi155.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-03-23T13:07:31.186 INFO:teuthology.orchestra.run.smithi155.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-03-23T13:07:31.212 INFO:teuthology.orchestra.run.smithi155.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-23T13:07:31.237 INFO:teuthology.orchestra.run.smithi155.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-03-23T13:07:31.262 INFO:teuthology.orchestra.run.smithi155.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-03-23T13:07:31.287 INFO:teuthology.orchestra.run.smithi155.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-23T13:07:31.313 INFO:teuthology.orchestra.run.smithi155.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-03-23T13:07:31.338 INFO:teuthology.orchestra.run.smithi155.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-03-23T13:07:31.363 INFO:teuthology.orchestra.run.smithi155.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-03-23T13:07:31.388 INFO:teuthology.orchestra.run.smithi155.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-23T13:07:31.414 INFO:teuthology.orchestra.run.smithi155.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-23T13:07:31.414 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-03-23T13:07:31.439 INFO:teuthology.orchestra.run.smithi155.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-03-23T13:07:31.464 INFO:teuthology.orchestra.run.smithi155.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-23T13:07:31.490 INFO:teuthology.orchestra.run.smithi155.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-23T13:07:31.518 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-23T13:07:31.544 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-23T13:07:31.556 INFO:teuthology.orchestra.run.smithi155.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-03-23T13:07:31.575 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T13:07:31.584 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T13:07:31.598 INFO:teuthology.orchestra.run.smithi155.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-23T13:07:31.624 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-23T13:07:31.677 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-23T13:07:31.712 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-23T13:07:31.737 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-23T13:07:31.779 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-23T13:07:31.932 INFO:teuthology.orchestra.run.smithi155.stdout:(152/174): librbd1-19.0.0-1855.g21b3bd67.el9.x8 7.1 MB/s | 3.1 MB 00:00 2024-03-23T13:07:31.990 INFO:teuthology.orchestra.run.smithi155.stdout:(153/174): librados2-19.0.0-1855.g21b3bd67.el9. 6.4 MB/s | 3.4 MB 00:00 2024-03-23T13:07:32.041 INFO:teuthology.orchestra.run.smithi155.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 843 kB/s | 42 kB 00:00 2024-03-23T13:07:32.099 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-23T13:07:32.124 INFO:teuthology.orchestra.run.smithi155.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.9 MB/s | 248 kB 00:00 2024-03-23T13:07:32.166 INFO:teuthology.orchestra.run.smithi155.stdout:(156/174): NetworkManager-1.47.2-1.el9.x86_64.r 4.1 MB/s | 2.3 MB 00:00 2024-03-23T13:07:32.174 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T13:07:32.225 INFO:teuthology.orchestra.run.smithi155.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 626 kB/s | 36 kB 00:00 2024-03-23T13:07:32.283 INFO:teuthology.orchestra.run.smithi155.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 449 kB/s | 71 kB 00:00 2024-03-23T13:07:32.285 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T13:07:32.309 INFO:teuthology.orchestra.run.smithi155.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.2 MB/s | 192 kB 00:00 2024-03-23T13:07:32.316 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T13:07:32.367 INFO:teuthology.orchestra.run.smithi155.stdout:(160/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.2 MB/s | 1.8 MB 00:00 2024-03-23T13:07:32.393 INFO:teuthology.orchestra.run.smithi155.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 276 kB/s | 23 kB 00:00 2024-03-23T13:07:32.407 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T13:07:32.418 INFO:teuthology.orchestra.run.smithi155.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.9 MB/s | 256 kB 00:00 2024-03-23T13:07:32.437 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T13:07:32.443 INFO:teuthology.orchestra.run.smithi155.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-03-23T13:07:32.468 INFO:teuthology.orchestra.run.smithi155.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-23T13:07:32.473 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-23T13:07:32.494 INFO:teuthology.orchestra.run.smithi155.stdout:(165/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-23T13:07:32.511 INFO:teuthology.orchestra.run.smithi155.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 948 kB/s | 64 kB 00:00 2024-03-23T13:07:32.528 INFO:teuthology.orchestra.run.smithi155.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 487 kB/s | 29 kB 00:00 2024-03-23T13:07:32.651 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-23T13:07:32.700 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-23T13:07:32.721 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T13:07:32.807 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T13:07:32.830 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T13:07:32.845 INFO:teuthology.orchestra.run.smithi155.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.5 MB/s | 1.2 MB 00:00 2024-03-23T13:07:32.877 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T13:07:32.917 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-23T13:07:32.945 INFO:teuthology.orchestra.run.smithi155.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.0 MB/s | 2.2 MB 00:00 2024-03-23T13:07:32.948 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T13:07:32.963 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T13:07:32.987 INFO:teuthology.orchestra.run.smithi155.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 751 kB/s | 31 kB 00:00 2024-03-23T13:07:33.012 INFO:teuthology.orchestra.run.smithi155.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 2.7 MB/s | 465 kB 00:00 2024-03-23T13:07:33.063 INFO:teuthology.orchestra.run.smithi155.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.3 MB/s | 2.3 MB 00:00 2024-03-23T13:07:33.123 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T13:07:33.129 INFO:teuthology.orchestra.run.smithi155.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 637 kB/s | 74 kB 00:00 2024-03-23T13:07:33.154 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T13:07:33.194 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-23T13:07:33.254 INFO:teuthology.orchestra.run.smithi155.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.0 MB/s | 1.1 MB 00:00 2024-03-23T13:07:33.259 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-23T13:07:33.259 INFO:teuthology.orchestra.run.smithi155.stdout:Total 16 MB/s | 224 MB 00:13 2024-03-23T13:07:33.299 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-23T13:07:33.371 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T13:07:33.431 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T13:07:33.499 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T13:07:33.574 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T13:07:33.748 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-23T13:07:33.785 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T13:07:33.897 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-23T13:07:33.998 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-23T13:07:34.023 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-23T13:07:34.023 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-23T13:07:34.037 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-23T13:07:34.063 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-23T13:07:34.098 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-23T13:07:34.165 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-23T13:07:34.218 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-23T13:07:34.257 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-23T13:07:34.321 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-23T13:07:34.355 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-23T13:07:34.400 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-23T13:07:34.432 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-23T13:07:34.460 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-23T13:07:34.529 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-23T13:07:34.564 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 39/198 2024-03-23T13:07:34.607 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 40/198 2024-03-23T13:07:34.636 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 41/198 2024-03-23T13:07:34.704 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-23T13:07:34.751 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-23T13:07:34.824 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-23T13:07:34.958 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-23T13:07:35.003 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-23T13:07:35.522 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-23T13:07:35.598 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-23T13:07:35.678 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-23T13:07:35.718 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T13:07:35.726 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T13:07:35.740 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-23T13:07:35.740 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-23T13:07:35.758 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-23T13:07:35.807 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-23T13:07:35.844 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 53/198 2024-03-23T13:07:35.874 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-23T13:07:36.107 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-23T13:07:36.270 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-23T13:07:36.317 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-23T13:07:36.361 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-23T13:07:36.504 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-23T13:07:36.555 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-23T13:07:36.610 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-23T13:07:36.683 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-23T13:07:36.731 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-23T13:07:36.767 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-23T13:07:36.813 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-23T13:07:36.858 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-23T13:07:36.901 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-23T13:07:36.944 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-23T13:07:37.077 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-23T13:07:37.300 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-23T13:07:37.335 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-23T13:07:37.376 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-23T13:07:37.473 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-23T13:07:37.509 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-23T13:07:37.549 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-23T13:07:37.575 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-23T13:07:37.599 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-23T13:07:37.663 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-23T13:07:37.784 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-23T13:07:37.892 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-23T13:07:37.932 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-23T13:07:37.980 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-23T13:07:38.036 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-23T13:07:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-23T13:07:38.187 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-23T13:07:38.219 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-23T13:07:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-23T13:07:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T13:07:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-23T13:07:38.270 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T13:07:38.319 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-23T13:07:38.373 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-23T13:07:38.405 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-23T13:07:38.405 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-23T13:07:38.430 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-23T13:07:38.442 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-23T13:07:38.470 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-23T13:07:38.474 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-23T13:07:38.505 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-23T13:07:38.551 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-23T13:07:38.592 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-23T13:07:38.631 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-23T13:07:38.663 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-23T13:07:38.693 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-23T13:07:38.769 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-23T13:07:38.811 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-23T13:07:38.845 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-23T13:07:38.847 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-23T13:07:38.884 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T13:07:38.886 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-23T13:07:38.921 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-23T13:07:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T13:07:39.009 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-23T13:07:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T13:07:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T13:07:39.140 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-23T13:07:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T13:07:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-23T13:07:39.377 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-23T13:07:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-23T13:07:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-23T13:07:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T13:07:39.548 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-23T13:07:39.586 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T13:07:39.614 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-23T13:07:39.647 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T13:07:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T13:07:39.712 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-23T13:07:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-23T13:07:39.759 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-23T13:07:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T13:07:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T13:07:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T13:07:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T13:07:40.030 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-23T13:07:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-23T13:07:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T13:07:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T13:07:40.474 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-23T13:07:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T13:07:40.523 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-23T13:07:40.556 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-23T13:07:40.572 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T13:07:40.604 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-23T13:07:40.684 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-23T13:07:40.724 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-23T13:07:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T13:07:40.891 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-23T13:07:40.921 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-23T13:07:40.998 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-23T13:07:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-23T13:07:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-23T13:07:41.302 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-23T13:07:41.333 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-23T13:07:41.376 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-23T13:07:41.450 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-23T13:07:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-23T13:07:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-23T13:07:41.615 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-23T13:07:41.636 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-23T13:07:41.662 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-23T13:07:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-23T13:07:41.740 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-23T13:07:41.742 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-23T13:07:41.767 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-23T13:07:41.839 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-23T13:07:41.872 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 39/198 2024-03-23T13:07:41.915 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 40/198 2024-03-23T13:07:41.944 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 41/198 2024-03-23T13:07:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-23T13:07:42.059 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-23T13:07:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-23T13:07:42.271 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-23T13:07:42.329 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-23T13:07:42.603 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-23T13:07:42.717 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-23T13:07:42.817 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-23T13:07:42.848 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T13:07:42.858 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T13:07:42.879 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-23T13:07:42.898 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-23T13:07:42.951 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-23T13:07:42.953 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-23T13:07:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-23T13:07:42.985 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-23T13:07:43.011 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-23T13:07:43.029 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-23T13:07:43.039 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-23T13:07:43.053 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-23T13:07:43.059 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-23T13:07:43.084 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T13:07:43.091 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-23T13:07:43.093 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T13:07:43.118 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-23T13:07:43.134 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-23T13:07:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-23T13:07:43.220 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 53/198 2024-03-23T13:07:43.251 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-23T13:07:43.379 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-23T13:07:43.427 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-23T13:07:43.428 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T13:07:43.457 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T13:07:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-23T13:07:43.530 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T13:07:43.548 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T13:07:43.561 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 125/198 2024-03-23T13:07:43.571 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T13:07:43.597 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-23T13:07:43.662 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T13:07:43.683 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-23T13:07:43.692 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T13:07:43.728 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-23T13:07:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-23T13:07:43.900 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-23T13:07:43.964 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-23T13:07:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-23T13:07:43.992 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T13:07:44.054 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-23T13:07:44.085 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T13:07:44.097 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T13:07:44.121 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T13:07:44.123 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-23T13:07:44.156 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T13:07:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-23T13:07:44.178 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 129/198 2024-03-23T13:07:44.185 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T13:07:44.216 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-23T13:07:44.230 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-23T13:07:44.239 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T13:07:44.254 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T13:07:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-23T13:07:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-23T13:07:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-23T13:07:44.422 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T13:07:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-23T13:07:44.453 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T13:07:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-23T13:07:44.485 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-23T13:07:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-23T13:07:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-23T13:07:44.591 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-23T13:07:44.688 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T13:07:44.692 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-23T13:07:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-23T13:07:44.748 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T13:07:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-23T13:07:44.816 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-23T13:07:44.825 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T13:07:44.883 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T13:07:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-23T13:07:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-23T13:07:44.982 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-23T13:07:45.007 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-23T13:07:45.031 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-23T13:07:45.109 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T13:07:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-23T13:07:45.219 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-23T13:07:45.252 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-23T13:07:45.310 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-23T13:07:45.359 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-23T13:07:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-23T13:07:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-23T13:07:45.504 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-23T13:07:45.527 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-23T13:07:45.634 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-23T13:07:45.685 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-23T13:07:45.692 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-23T13:07:45.736 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-23T13:07:45.759 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-23T13:07:45.812 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-23T13:07:45.850 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-23T13:07:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-23T13:07:45.916 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-23T13:07:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-23T13:07:45.949 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-23T13:07:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-23T13:07:45.989 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-23T13:07:45.992 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-23T13:07:46.017 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-23T13:07:46.025 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-23T13:07:46.051 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-23T13:07:46.053 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-23T13:07:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-23T13:07:46.122 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-23T13:07:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-23T13:07:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-23T13:07:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 39/198 2024-03-23T13:07:46.201 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 40/198 2024-03-23T13:07:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-23T13:07:46.231 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 41/198 2024-03-23T13:07:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-23T13:07:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-23T13:07:46.299 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-23T13:07:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-23T13:07:46.344 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-23T13:07:46.363 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T13:07:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-23T13:07:46.399 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T13:07:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-23T13:07:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-23T13:07:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-23T13:07:46.561 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-23T13:07:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-23T13:07:46.614 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-23T13:07:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-23T13:07:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-23T13:07:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-23T13:07:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-23T13:07:46.937 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T13:07:46.969 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T13:07:47.049 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T13:07:47.050 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T13:07:47.050 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T13:07:47.050 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:07:47.070 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T13:07:47.135 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-23T13:07:47.218 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-23T13:07:47.307 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-23T13:07:47.355 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T13:07:47.364 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T13:07:47.395 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-23T13:07:47.435 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-23T13:07:47.473 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 53/198 2024-03-23T13:07:47.504 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-23T13:07:47.577 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-23T13:07:47.617 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-23T13:07:47.651 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-23T13:07:47.700 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-23T13:07:47.725 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-23T13:07:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-23T13:07:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-23T13:07:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-23T13:07:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-23T13:07:48.124 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-23T13:07:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-23T13:07:48.207 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-23T13:07:48.253 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-23T13:07:48.403 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-23T13:07:48.445 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-23T13:07:48.500 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-23T13:07:48.581 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-23T13:07:48.613 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-23T13:07:48.650 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-23T13:07:48.696 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-23T13:07:48.732 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-23T13:07:48.775 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-23T13:07:48.817 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-23T13:07:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-23T13:07:48.953 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-23T13:07:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-23T13:07:48.994 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-23T13:07:49.025 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-23T13:07:49.066 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-23T13:07:49.161 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-23T13:07:49.208 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-23T13:07:49.248 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-23T13:07:49.274 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-23T13:07:49.298 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-23T13:07:49.371 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-23T13:07:49.500 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-23T13:07:49.599 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-23T13:07:49.640 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-23T13:07:49.689 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-23T13:07:49.745 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-23T13:07:49.930 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-23T13:07:49.986 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-23T13:07:50.115 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-23T13:07:50.160 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-23T13:07:50.191 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-23T13:07:50.222 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-23T13:07:50.230 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-23T13:07:50.250 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-23T13:07:50.285 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-23T13:07:50.314 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-23T13:07:50.324 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-23T13:07:50.345 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-23T13:07:50.355 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-23T13:07:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-23T13:07:50.403 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-23T13:07:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-23T13:07:50.470 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-23T13:07:50.513 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-23T13:07:50.547 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-23T13:07:50.588 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-23T13:07:50.623 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-23T13:07:50.711 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-23T13:07:50.732 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-23T13:07:50.759 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T13:07:50.793 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-23T13:07:50.837 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-23T13:07:50.850 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T13:07:50.868 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-23T13:07:50.880 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 125/198 2024-03-23T13:07:50.917 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-23T13:07:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-23T13:07:51.023 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-23T13:07:51.078 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-23T13:07:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-23T13:07:51.429 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T13:07:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T13:07:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 129/198 2024-03-23T13:07:51.807 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-23T13:07:51.851 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-23T13:07:51.884 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-23T13:07:51.933 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-23T13:07:52.020 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-23T13:07:52.052 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-23T13:07:52.230 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-23T13:07:52.271 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-23T13:07:52.344 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-23T13:07:52.997 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-23T13:07:53.107 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-23T13:07:53.834 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T13:07:53.869 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T13:07:54.337 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-23T13:07:54.419 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-23T13:07:54.433 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T13:07:54.450 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-23T13:07:54.472 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T13:07:54.480 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-23T13:07:54.508 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-23T13:07:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T13:07:54.563 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T13:07:54.563 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T13:07:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:07:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T13:07:54.819 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-23T13:07:54.847 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T13:07:54.929 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T13:07:54.960 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 125/198 2024-03-23T13:07:54.996 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-23T13:07:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-23T13:07:55.483 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T13:07:55.538 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T13:07:55.568 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 129/198 2024-03-23T13:07:57.714 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T13:07:57.757 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T13:07:58.298 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T13:07:58.335 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T13:07:58.400 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T13:07:58.400 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T13:07:58.400 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T13:07:58.400 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:07:58.486 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T13:07:58.806 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T13:07:58.807 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /sys 2024-03-23T13:07:58.807 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /proc 2024-03-23T13:07:58.807 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /mnt 2024-03-23T13:07:58.807 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /var/tmp 2024-03-23T13:07:58.807 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /home 2024-03-23T13:07:58.807 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /root 2024-03-23T13:07:58.807 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /tmp 2024-03-23T13:07:58.807 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:07:58.976 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T13:07:58.994 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T13:07:58.994 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:07:58.994 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T13:07:58.994 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:00.475 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T13:08:00.501 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T13:08:00.501 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:00.501 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T13:08:00.501 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T13:08:00.501 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T13:08:00.501 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:00.734 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T13:08:00.757 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T13:08:00.757 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:00.757 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T13:08:00.757 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T13:08:00.757 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T13:08:00.757 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:01.202 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T13:08:01.224 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T13:08:01.224 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:01.224 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T13:08:01.225 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T13:08:01.225 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T13:08:01.225 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:01.320 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-23T13:08:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T13:08:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T13:08:01.807 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 139/198 2024-03-23T13:08:01.868 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 140/198 2024-03-23T13:08:01.899 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T13:08:02.131 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T13:08:02.167 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T13:08:03.138 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T13:08:03.175 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T13:08:03.276 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T13:08:03.439 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 144/198 2024-03-23T13:08:03.470 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T13:08:03.493 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T13:08:03.493 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:03.493 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T13:08:03.493 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T13:08:03.493 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T13:08:03.493 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:03.537 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T13:08:03.548 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T13:08:03.592 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 147/198 2024-03-23T13:08:04.450 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-23T13:08:04.488 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T13:08:04.511 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T13:08:04.511 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:04.511 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T13:08:04.511 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T13:08:04.512 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T13:08:04.512 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:04.555 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T13:08:04.578 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T13:08:04.578 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:04.578 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T13:08:04.578 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:04.623 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T13:08:04.645 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T13:08:04.645 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:04.645 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T13:08:04.645 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T13:08:04.645 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T13:08:04.645 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:04.965 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T13:08:04.988 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T13:08:04.988 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:04.988 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T13:08:04.988 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T13:08:04.988 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T13:08:04.988 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:07.151 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T13:08:07.152 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-03-23T13:08:07.152 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-03-23T13:08:07.152 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-03-23T13:08:07.152 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-03-23T13:08:07.152 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-03-23T13:08:07.152 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-03-23T13:08:07.152 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-03-23T13:08:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T13:08:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T13:08:07.338 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:07.338 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T13:08:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T13:08:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T13:08:08.918 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:08.918 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T13:08:08.918 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T13:08:08.918 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T13:08:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T13:08:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T13:08:09.174 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:09.174 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T13:08:09.174 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T13:08:09.174 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T13:08:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:09.421 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T13:08:09.468 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 154/198 2024-03-23T13:08:09.499 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 155/198 2024-03-23T13:08:09.526 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-23T13:08:09.635 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T13:08:09.657 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T13:08:09.657 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:09.658 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T13:08:09.658 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T13:08:09.658 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T13:08:09.658 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:09.705 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 157/198 2024-03-23T13:08:09.754 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-23T13:08:09.816 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-23T13:08:09.865 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 159/198 2024-03-23T13:08:09.904 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-23T13:08:09.949 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-23T13:08:09.984 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-23T13:08:10.016 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 163/198 2024-03-23T13:08:10.048 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-23T13:08:10.078 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-23T13:08:10.123 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-23T13:08:10.155 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T13:08:10.175 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-23T13:08:10.186 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T13:08:10.209 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-23T13:08:10.215 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T13:08:10.215 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-03-23T13:08:10.215 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-03-23T13:08:10.215 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-03-23T13:08:10.215 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-03-23T13:08:10.216 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-03-23T13:08:10.216 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /root 2024-03-23T13:08:10.216 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-03-23T13:08:10.216 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:10.218 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 139/198 2024-03-23T13:08:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 140/198 2024-03-23T13:08:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T13:08:10.393 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T13:08:10.408 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T13:08:10.408 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:10.408 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T13:08:10.408 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:10.551 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T13:08:10.577 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T13:08:10.788 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T13:08:10.834 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T13:08:10.854 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T13:08:11.591 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T13:08:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T13:08:11.663 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 170/198 2024-03-23T13:08:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T13:08:11.745 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-23T13:08:11.785 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-23T13:08:11.866 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-23T13:08:11.889 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 144/198 2024-03-23T13:08:11.899 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-23T13:08:11.906 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T13:08:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T13:08:11.923 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-23T13:08:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T13:08:11.932 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:11.932 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T13:08:11.932 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T13:08:11.932 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T13:08:11.932 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:11.943 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T13:08:11.943 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:11.943 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T13:08:11.944 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T13:08:11.944 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T13:08:11.944 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:11.949 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-23T13:08:11.975 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-23T13:08:11.975 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T13:08:11.988 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T13:08:12.000 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T13:08:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 147/198 2024-03-23T13:08:12.063 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T13:08:12.092 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-23T13:08:12.093 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T13:08:12.156 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T13:08:12.179 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T13:08:12.179 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:12.179 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T13:08:12.179 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T13:08:12.179 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T13:08:12.179 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:12.185 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T13:08:12.211 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-23T13:08:12.245 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-23T13:08:12.270 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-23T13:08:12.295 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-23T13:08:12.349 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-23T13:08:12.384 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-23T13:08:12.416 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-23T13:08:12.441 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-23T13:08:12.467 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-23T13:08:12.492 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-23T13:08:12.517 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-23T13:08:12.524 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T13:08:12.524 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T13:08:12.632 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T13:08:12.654 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T13:08:12.655 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:12.655 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T13:08:12.655 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T13:08:12.655 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T13:08:12.655 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:12.703 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T13:08:12.704 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T13:08:12.743 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-23T13:08:12.786 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T13:08:12.811 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-23T13:08:12.843 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-23T13:08:12.870 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-23T13:08:12.895 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-23T13:08:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-23T13:08:12.959 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T13:08:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T13:08:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T13:08:12.998 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:12.998 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T13:08:12.998 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T13:08:12.998 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T13:08:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T13:08:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T13:08:13.063 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:13.063 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T13:08:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T13:08:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T13:08:13.139 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:13.139 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T13:08:13.140 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T13:08:13.140 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T13:08:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:13.152 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T13:08:13.183 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T13:08:13.214 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 139/198 2024-03-23T13:08:13.275 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 140/198 2024-03-23T13:08:13.297 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T13:08:13.460 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T13:08:13.482 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T13:08:13.482 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:13.482 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T13:08:13.482 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T13:08:13.482 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T13:08:13.483 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:13.531 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T13:08:13.565 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T13:08:14.536 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T13:08:14.565 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T13:08:14.667 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T13:08:14.828 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 144/198 2024-03-23T13:08:14.860 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T13:08:14.882 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T13:08:14.883 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:14.883 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T13:08:14.883 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T13:08:14.883 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T13:08:14.883 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:14.927 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T13:08:14.938 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T13:08:14.991 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 147/198 2024-03-23T13:08:15.855 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-23T13:08:15.896 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T13:08:15.919 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T13:08:15.919 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:15.919 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T13:08:15.919 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T13:08:15.919 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T13:08:15.919 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:15.963 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T13:08:15.985 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T13:08:15.985 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:15.986 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T13:08:15.986 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:16.039 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T13:08:16.061 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T13:08:16.061 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:16.061 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T13:08:16.061 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T13:08:16.062 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T13:08:16.062 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:16.373 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T13:08:16.396 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T13:08:16.396 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T13:08:16.396 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T13:08:16.396 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T13:08:16.396 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T13:08:16.396 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:17.723 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T13:08:17.723 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 1/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 3/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 4/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 5/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 6/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 7/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 8/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 9/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 13/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 14/198 2024-03-23T13:08:17.724 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 15/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 17/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 18/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 19/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 20/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 21/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 22/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 23/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 24/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 25/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 26/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 27/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 28/198 2024-03-23T13:08:17.725 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 29/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 30/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 31/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 32/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 33/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 34/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 35/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 36/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 37/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 38/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 39/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-23T13:08:17.726 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-23T13:08:17.727 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-23T13:08:17.728 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-23T13:08:17.729 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-23T13:08:17.730 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-23T13:08:17.731 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-23T13:08:17.732 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-23T13:08:17.733 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-23T13:08:17.734 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-23T13:08:17.735 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-23T13:08:17.736 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-23T13:08:17.737 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-23T13:08:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T13:08:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 154/198 2024-03-23T13:08:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 155/198 2024-03-23T13:08:18.289 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-23T13:08:18.480 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 157/198 2024-03-23T13:08:18.601 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-23T13:08:18.649 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 159/198 2024-03-23T13:08:18.702 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-23T13:08:18.764 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-23T13:08:18.808 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-23T13:08:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 163/198 2024-03-23T13:08:18.881 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-23T13:08:18.919 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-23T13:08:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-23T13:08:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-23T13:08:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-23T13:08:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T13:08:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T13:08:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T13:08:20.671 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout:Upgraded: 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T13:08:20.672 INFO:teuthology.orchestra.run.smithi130.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T13:08:20.673 INFO:teuthology.orchestra.run.smithi130.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.674 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.675 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T13:08:20.676 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: jq-1.6-16.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T13:08:20.677 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: lua-5.4.4-4.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-23T13:08:20.678 INFO:teuthology.orchestra.run.smithi130.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T13:08:20.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T13:08:20.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T13:08:20.681 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T13:08:20.682 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T13:08:20.683 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T13:08:20.684 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T13:08:20.685 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T13:08:20.686 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 170/198 2024-03-23T13:08:20.686 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T13:08:20.686 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-03-23T13:08:20.686 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-03-23T13:08:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-23T13:08:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-23T13:08:20.811 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T13:08:20.852 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 154/198 2024-03-23T13:08:20.883 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 155/198 2024-03-23T13:08:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-23T13:08:20.910 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-23T13:08:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-23T13:08:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-23T13:08:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-23T13:08:21.017 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-23T13:08:21.018 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T13:08:21.092 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 157/198 2024-03-23T13:08:21.124 DEBUG:teuthology.parallel:result is None 2024-03-23T13:08:21.131 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T13:08:21.160 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-23T13:08:21.160 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T13:08:21.212 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-23T13:08:21.245 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T13:08:21.253 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 159/198 2024-03-23T13:08:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-23T13:08:21.291 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-23T13:08:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-23T13:08:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-23T13:08:21.344 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-23T13:08:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-23T13:08:21.387 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-23T13:08:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-23T13:08:21.411 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 163/198 2024-03-23T13:08:21.444 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-23T13:08:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-23T13:08:21.473 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-23T13:08:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-23T13:08:21.510 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-23T13:08:21.536 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-23T13:08:21.536 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-23T13:08:21.561 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-23T13:08:21.586 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-23T13:08:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-23T13:08:21.593 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T13:08:21.593 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T13:08:21.630 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-23T13:08:21.772 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T13:08:21.773 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T13:08:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T13:08:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-23T13:08:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-23T13:08:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-23T13:08:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-23T13:08:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T13:08:22.261 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T13:08:22.304 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T13:08:22.323 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T13:08:23.163 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 170/198 2024-03-23T13:08:23.249 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-23T13:08:23.290 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-23T13:08:23.372 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-23T13:08:23.412 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-23T13:08:23.437 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-23T13:08:23.462 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-23T13:08:23.489 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-23T13:08:23.489 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T13:08:23.560 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T13:08:23.590 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-23T13:08:23.590 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T13:08:23.658 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T13:08:23.684 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-23T13:08:23.717 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-23T13:08:23.742 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-23T13:08:23.759 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-23T13:08:23.792 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-23T13:08:23.832 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-23T13:08:23.864 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-23T13:08:23.890 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-23T13:08:23.915 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-23T13:08:23.949 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-23T13:08:23.974 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-23T13:08:23.980 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T13:08:23.980 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T13:08:24.168 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T13:08:24.168 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T13:08:24.243 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T13:08:24.268 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-23T13:08:24.300 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-23T13:08:24.327 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-23T13:08:24.352 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-23T13:08:24.417 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T13:08:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T13:08:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 1/198 2024-03-23T13:08:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2/198 2024-03-23T13:08:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 3/198 2024-03-23T13:08:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 4/198 2024-03-23T13:08:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 5/198 2024-03-23T13:08:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 6/198 2024-03-23T13:08:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 7/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 8/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 9/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 13/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 14/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 15/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 17/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 18/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 19/198 2024-03-23T13:08:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 20/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 21/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 22/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 23/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 24/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 25/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 26/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 27/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 28/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 29/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 30/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 31/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 32/198 2024-03-23T13:08:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 33/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 34/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 35/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 36/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 37/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 38/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 39/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-23T13:08:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-23T13:08:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-23T13:08:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-23T13:08:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-23T13:08:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-23T13:08:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-23T13:08:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-23T13:08:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T13:08:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-23T13:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-23T13:08:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-23T13:08:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-23T13:08:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-23T13:08:29.289 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T13:08:29.289 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 1/198 2024-03-23T13:08:29.289 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2/198 2024-03-23T13:08:29.289 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 3/198 2024-03-23T13:08:29.289 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 4/198 2024-03-23T13:08:29.289 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 5/198 2024-03-23T13:08:29.289 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 6/198 2024-03-23T13:08:29.289 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 7/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 8/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 9/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 13/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 14/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 15/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 17/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 18/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 19/198 2024-03-23T13:08:29.290 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 20/198 2024-03-23T13:08:29.291 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 21/198 2024-03-23T13:08:29.291 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 22/198 2024-03-23T13:08:29.291 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 23/198 2024-03-23T13:08:29.291 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 24/198 2024-03-23T13:08:29.291 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 25/198 2024-03-23T13:08:29.291 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 26/198 2024-03-23T13:08:29.306 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 27/198 2024-03-23T13:08:29.306 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 28/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 29/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 30/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 31/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 32/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 33/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 34/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 35/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 36/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 37/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 38/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 39/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-23T13:08:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-23T13:08:29.308 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-23T13:08:29.309 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-23T13:08:29.310 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-23T13:08:29.311 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-23T13:08:29.312 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-23T13:08:29.313 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-23T13:08:29.314 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-23T13:08:29.315 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-23T13:08:29.316 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-23T13:08:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-23T13:08:29.318 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-23T13:08:29.319 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-23T13:08:29.319 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-23T13:08:29.319 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-23T13:08:30.036 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-23T13:08:30.036 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:30.036 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T13:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-23T13:08:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: jq-1.6-16.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T13:08:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T13:08:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-23T13:08:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T13:08:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T13:08:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T13:08:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-23T13:08:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-23T13:08:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-23T13:08:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T13:08:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T13:08:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T13:08:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T13:08:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T13:08:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T13:08:30.050 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T13:08:30.050 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:08:30.050 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-23T13:08:30.529 DEBUG:teuthology.parallel:result is None 2024-03-23T13:08:32.266 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-23T13:08:32.266 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:32.266 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-03-23T13:08:32.266 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:32.266 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T13:08:32.267 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.268 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:32.269 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T13:08:32.270 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: jq-1.6-16.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T13:08:32.271 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: lua-5.4.4-4.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T13:08:32.272 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T13:08:32.273 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T13:08:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T13:08:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T13:08:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T13:08:32.277 INFO:teuthology.orchestra.run.smithi155.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T13:08:32.278 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T13:08:32.278 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T13:08:32.278 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T13:08:32.278 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T13:08:32.278 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T13:08:32.278 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T13:08:32.278 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:08:32.278 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-23T13:08:32.751 DEBUG:teuthology.parallel:result is None 2024-03-23T13:08:32.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:08:32.888 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T13:08:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:19.0.0-1855.g21b3bd67.el9 2024-03-23T13:08:32.914 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1855.g21b3bd67.el9 2024-03-23T13:08:32.914 INFO:teuthology.task.install:The correct ceph version 19.0.0-1855.g21b3bd67 is installed. 2024-03-23T13:08:32.915 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:08:33.125 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T13:08:33.150 INFO:teuthology.orchestra.run.smithi130.stdout:19.0.0-1855.g21b3bd67.el9 2024-03-23T13:08:33.150 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1855.g21b3bd67.el9 2024-03-23T13:08:33.150 INFO:teuthology.task.install:The correct ceph version 19.0.0-1855.g21b3bd67 is installed. 2024-03-23T13:08:33.152 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:08:33.365 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T13:08:33.390 INFO:teuthology.orchestra.run.smithi155.stdout:19.0.0-1855.g21b3bd67.el9 2024-03-23T13:08:33.390 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1855.g21b3bd67.el9 2024-03-23T13:08:33.391 INFO:teuthology.task.install:The correct ceph version 19.0.0-1855.g21b3bd67 is installed. 2024-03-23T13:08:33.392 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-23T13:08:33.392 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:08:33.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T13:08:33.427 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:08:33.427 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T13:08:33.461 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:08:33.462 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T13:08:33.499 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-23T13:08:33.499 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:08:33.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T13:08:33.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T13:08:33.625 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:08:33.626 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T13:08:33.661 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T13:08:33.745 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:08:33.746 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T13:08:33.782 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T13:08:33.857 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-23T13:08:33.857 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:08:33.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T13:08:33.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T13:08:33.961 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:08:33.962 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T13:08:33.989 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T13:08:34.059 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:08:34.059 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T13:08:34.094 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T13:08:34.178 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-23T13:08:34.179 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:08:34.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T13:08:34.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T13:08:34.284 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:08:34.285 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T13:08:34.312 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T13:08:34.380 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:08:34.381 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T13:08:34.413 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T13:08:34.485 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-23T13:08:34.601 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'subvols': {'create': 2}} 2024-03-23T13:08:34.601 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:08:34.602 INFO:tasks.cephadm:Cluster fsid is 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:08:34.602 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-23T13:08:34.602 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.7', 'mon.b': '172.21.15.130', 'mon.c': '172.21.15.155'} 2024-03-23T13:08:34.602 INFO:tasks.cephadm:First mon is mon.a on smithi007 2024-03-23T13:08:34.602 INFO:tasks.cephadm:First mgr is x 2024-03-23T13:08:34.603 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-23T13:08:34.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname $(hostname -s) 2024-03-23T13:08:34.635 DEBUG:teuthology.orchestra.run.smithi130:> sudo hostname $(hostname -s) 2024-03-23T13:08:34.664 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-03-23T13:08:34.697 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-23T13:08:34.697 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:08:34.928 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77900/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1855-g21b3bd67', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1855.g21b3bd67'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-23 04:48:23.862772', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240323.003144-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-23T13:08:35.155 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20240323.003144-squid, sha1 21b3bd672163092b4e542f772797fcba02d3d4de from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:08:35.157 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/x86_64/flavors/default/cephadm 2024-03-23T13:08:35.157 DEBUG:teuthology.orchestra.run.smithi007:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T13:08:35.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-r--r--. 1 ubuntu ubuntu 766828 Mar 23 13:08 /home/ubuntu/cephtest/cephadm 2024-03-23T13:08:35.790 DEBUG:teuthology.orchestra.run.smithi130:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T13:08:36.313 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-r--r--. 1 ubuntu ubuntu 766828 Mar 23 13:08 /home/ubuntu/cephtest/cephadm 2024-03-23T13:08:36.313 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T13:08:36.930 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r--. 1 ubuntu ubuntu 766828 Mar 23 13:08 /home/ubuntu/cephtest/cephadm 2024-03-23T13:08:36.930 DEBUG:teuthology.orchestra.run.smithi007:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T13:08:36.949 DEBUG:teuthology.orchestra.run.smithi130:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T13:08:36.967 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T13:08:37.001 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de on all hosts... 2024-03-23T13:08:37.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de pull 2024-03-23T13:08:37.004 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de pull 2024-03-23T13:08:37.009 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de pull 2024-03-23T13:08:37.806 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de... 2024-03-23T13:08:37.910 INFO:teuthology.orchestra.run.smithi130.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de... 2024-03-23T13:08:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de... 2024-03-23T13:09:08.557 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-03-23T13:09:08.558 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)", 2024-03-23T13:09:08.558 INFO:teuthology.orchestra.run.smithi130.stdout: "image_id": "553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12", 2024-03-23T13:09:08.558 INFO:teuthology.orchestra.run.smithi130.stdout: "repo_digests": [ 2024-03-23T13:09:08.558 INFO:teuthology.orchestra.run.smithi130.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919" 2024-03-23T13:09:08.558 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2024-03-23T13:09:08.558 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-03-23T13:09:28.186 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-23T13:09:28.186 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)", 2024-03-23T13:09:28.187 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12", 2024-03-23T13:09:28.187 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2024-03-23T13:09:28.187 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919" 2024-03-23T13:09:28.187 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-03-23T13:09:28.187 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-23T13:09:28.613 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:09:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)", 2024-03-23T13:09:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "image_id": "553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12", 2024-03-23T13:09:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "repo_digests": [ 2024-03-23T13:09:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919" 2024-03-23T13:09:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-23T13:09:28.613 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:09:28.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph 2024-03-23T13:09:28.692 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph 2024-03-23T13:09:28.728 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-03-23T13:09:28.760 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /etc/ceph 2024-03-23T13:09:28.786 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 777 /etc/ceph 2024-03-23T13:09:28.817 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-03-23T13:09:28.845 INFO:tasks.cephadm:Writing seed config... 2024-03-23T13:09:28.846 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-23T13:09:28.846 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-23T13:09:28.846 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-23T13:09:28.846 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-23T13:09:28.846 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-23T13:09:28.846 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-23T13:09:28.846 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-23T13:09:28.846 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-23T13:09:28.846 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-23T13:09:28.846 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-23T13:09:28.847 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-23T13:09:28.848 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-23T13:09:28.849 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-23T13:09:28.849 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-23T13:09:28.849 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-23T13:09:28.849 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-23T13:09:28.849 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-23T13:09:28.849 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-23T13:09:28.849 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-23T13:09:28.850 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:09:28.850 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-23T13:09:28.864 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 745f591a-e916-11ee-95cd-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-23T13:09:28.864 DEBUG:teuthology.orchestra.run.smithi007:mon.a> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.a.service 2024-03-23T13:09:28.907 DEBUG:teuthology.orchestra.run.smithi007:mgr.x> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.x.service 2024-03-23T13:09:28.949 INFO:tasks.cephadm:Bootstrapping... 2024-03-23T13:09:28.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de -v bootstrap --fsid 745f591a-e916-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.7 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-23T13:09:29.145 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-23T13:09:29.145 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de', '-v', 'bootstrap', '--fsid', '745f591a-e916-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.7', '--skip-admin-label'] 2024-03-23T13:09:29.170 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-23T13:09:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-23T13:09:29.170 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying podman|docker is present... 2024-03-23T13:09:29.193 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-23T13:09:29.193 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying lvm2 is present... 2024-03-23T13:09:29.194 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying time synchronization is in place... 2024-03-23T13:09:29.201 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-23T13:09:29.201 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-23T13:09:29.207 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-23T13:09:29.207 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-23T13:09:29.213 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-03-23T13:09:29.219 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-03-23T13:09:29.219 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-03-23T13:09:29.219 INFO:teuthology.orchestra.run.smithi007.stdout:Repeating the final host check... 2024-03-23T13:09:29.241 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-23T13:09:29.241 INFO:teuthology.orchestra.run.smithi007.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-23T13:09:29.241 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl is present 2024-03-23T13:09:29.241 INFO:teuthology.orchestra.run.smithi007.stdout:lvcreate is present 2024-03-23T13:09:29.250 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-23T13:09:29.251 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-23T13:09:29.253 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-23T13:09:29.253 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-23T13:09:29.259 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-03-23T13:09:29.265 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-03-23T13:09:29.265 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-03-23T13:09:29.265 INFO:teuthology.orchestra.run.smithi007.stdout:Host looks OK 2024-03-23T13:09:29.265 INFO:teuthology.orchestra.run.smithi007.stdout:Cluster fsid: 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:09:29.265 INFO:teuthology.orchestra.run.smithi007.stdout:Acquiring lock 139633604718544 on /run/cephadm/745f591a-e916-11ee-95cd-87774f69a715.lock 2024-03-23T13:09:29.266 INFO:teuthology.orchestra.run.smithi007.stdout:Lock 139633604718544 acquired on /run/cephadm/745f591a-e916-11ee-95cd-87774f69a715.lock 2024-03-23T13:09:29.266 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 3300 ... 2024-03-23T13:09:29.266 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 6789 ... 2024-03-23T13:09:29.266 INFO:teuthology.orchestra.run.smithi007.stdout:Base mon IP(s) is [172.21.15.7:3300, 172.21.15.7:6789], mon addrv is [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-03-23T13:09:29.269 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.7 metric 100 2024-03-23T13:09:29.270 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.7 metric 100 2024-03-23T13:09:29.272 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-23T13:09:29.272 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-23T13:09:29.272 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-23T13:09:29.275 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-23T13:09:29.275 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-23T13:09:29.275 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-23T13:09:29.275 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-23T13:09:29.275 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-03-23T13:09:29.275 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-23T13:09:29.275 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-03-23T13:09:29.275 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-03-23T13:09:29.276 INFO:teuthology.orchestra.run.smithi007.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-23T13:09:29.276 INFO:teuthology.orchestra.run.smithi007.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-23T13:09:29.277 INFO:teuthology.orchestra.run.smithi007.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de... 2024-03-23T13:09:31.375 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 2024-03-23T13:09:31.375 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de... 2024-03-23T13:09:31.375 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Getting image source signatures 2024-03-23T13:09:31.375 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:30dc77a9f705c02d26e1a1f9e2bfdf9d08cd4086b08b221e8a27d0dd2909924d 2024-03-23T13:09:31.375 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-23T13:09:31.375 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying config sha256:553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 2024-03-23T13:09:31.375 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-23T13:09:32.314 INFO:teuthology.orchestra.run.smithi007.stdout:ceph: stdout ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev) 2024-03-23T13:09:32.315 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph version: ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev) 2024-03-23T13:09:32.315 INFO:teuthology.orchestra.run.smithi007.stdout:Extracting ceph user uid/gid from container image... 2024-03-23T13:09:33.253 INFO:teuthology.orchestra.run.smithi007.stdout:stat: stdout 167 167 2024-03-23T13:09:33.253 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial keys... 2024-03-23T13:09:34.141 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQCN1P5lqPq3IBAAIl4wCjwYxrbwAg1l6C0RoQ== 2024-03-23T13:09:35.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQCO1P5lhmeOHBAABDe/6B5A5OzmPTa+euC4SQ== 2024-03-23T13:09:35.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQCP1P5lVa5tFhAAY8nUj9HjvEcwb7sZ56Y+/g== 2024-03-23T13:09:35.888 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial monmap... 2024-03-23T13:09:36.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-23T13:09:36.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-23T13:09:36.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:09:36.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-23T13:09:36.742 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool for a [v2:172.21.15.7:3300,v1:172.21.15.7:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-23T13:09:36.743 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-03-23T13:09:36.743 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: set fsid to 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:09:36.743 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-23T13:09:36.743 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:09:36.743 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mon... 2024-03-23T13:09:37.757 INFO:teuthology.orchestra.run.smithi007.stdout:create mon.a on 2024-03-23T13:09:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-23T13:09:38.107 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-23T13:09:38.272 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-745f591a-e916-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-745f591a-e916-11ee-95cd-87774f69a715.target. 2024-03-23T13:09:38.273 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-745f591a-e916-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-745f591a-e916-11ee-95cd-87774f69a715.target. 2024-03-23T13:09:38.520 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.a 2024-03-23T13:09:38.520 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-23T13:09:38.677 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-745f591a-e916-11ee-95cd-87774f69a715.target.wants/ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-745f591a-e916-11ee-95cd-87774f69a715@.service. 2024-03-23T13:09:39.189 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T13:09:39.189 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-23T13:09:39.196 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T13:09:39.196 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-23T13:09:39.196 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-23T13:09:39.196 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-03-23T13:09:39.197 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon to start... 2024-03-23T13:09:39.197 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon... 2024-03-23T13:09:40.255 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:40 smithi007 ceph-mon[31803]: from='client.? 172.21.15.7:0/2607859614' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-23T13:09:40.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout cluster: 2024-03-23T13:09:40.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout id: 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:09:40.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-23T13:09:40.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:09:40.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout services: 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.524408s) 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout data: 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pgs: 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.639+0000 7fe7e2ae8700 1 Processor -- start 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.640+0000 7fe7e2ae8700 1 -- start start 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.640+0000 7fe7e2ae8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc107360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.640+0000 7fe7e2ae8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc107930 con 0x7fe7dc106f40 2024-03-23T13:09:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.640+0000 7fe7e0884700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc107360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.640+0000 7fe7e0884700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc107360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57948/0 (socket says 172.21.15.7:57948) 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.640+0000 7fe7e0884700 1 -- 172.21.15.7:0/4080465570 learned_addr learned my addr 172.21.15.7:0/4080465570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.641+0000 7fe7e0884700 1 -- 172.21.15.7:0/4080465570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc107c60 con 0x7fe7dc106f40 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.641+0000 7fe7e0884700 1 --2- 172.21.15.7:0/4080465570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc107360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00098a0 tx=0x7fe7d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639fb679068c5869 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.641+0000 7fe7db7fe700 1 -- 172.21.15.7:0/4080465570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d0004160 con 0x7fe7dc106f40 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.641+0000 7fe7db7fe700 1 -- 172.21.15.7:0/4080465570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d00042c0 con 0x7fe7dc106f40 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.641+0000 7fe7db7fe700 1 -- 172.21.15.7:0/4080465570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d00045a0 con 0x7fe7dc106f40 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.641+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/4080465570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 msgr2=0x7fe7dc107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.641+0000 7fe7e2ae8700 1 --2- 172.21.15.7:0/4080465570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc107360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00098a0 tx=0x7fe7d0009bb0 comp rx=0 tx=0).stop 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.641+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/4080465570 shutdown_connections 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.641+0000 7fe7e2ae8700 1 --2- 172.21.15.7:0/4080465570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc107360 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.641+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/4080465570 >> 172.21.15.7:0/4080465570 conn(0x7fe7dc1017a0 msgr2=0x7fe7dc103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:40.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.642+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/4080465570 shutdown_connections 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.642+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/4080465570 wait complete. 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.642+0000 7fe7e2ae8700 1 Processor -- start 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.642+0000 7fe7e2ae8700 1 -- start start 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.642+0000 7fe7e2ae8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc19b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.642+0000 7fe7e2ae8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc19b590 con 0x7fe7dc106f40 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.642+0000 7fe7e0884700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc19b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.642+0000 7fe7e0884700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc19b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57950/0 (socket says 172.21.15.7:57950) 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.642+0000 7fe7e0884700 1 -- 172.21.15.7:0/2607859614 learned_addr learned my addr 172.21.15.7:0/2607859614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.643+0000 7fe7e0884700 1 -- 172.21.15.7:0/2607859614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d0009550 con 0x7fe7dc106f40 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.643+0000 7fe7e0884700 1 --2- 172.21.15.7:0/2607859614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc19b050 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0681b0 tx=0x7fe7d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.643+0000 7fe7d9ffb700 1 -- 172.21.15.7:0/2607859614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d0011630 con 0x7fe7dc106f40 2024-03-23T13:09:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.643+0000 7fe7d9ffb700 1 -- 172.21.15.7:0/2607859614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d0011790 con 0x7fe7dc106f40 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.643+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/2607859614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7dc19b790 con 0x7fe7dc106f40 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.643+0000 7fe7d9ffb700 1 -- 172.21.15.7:0/2607859614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d0011a30 con 0x7fe7dc106f40 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.643+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/2607859614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7dc19bc30 con 0x7fe7dc106f40 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.643+0000 7fe7d9ffb700 1 -- 172.21.15.7:0/2607859614 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe7d0022070 con 0x7fe7dc106f40 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.643+0000 7fe7d9ffb700 1 -- 172.21.15.7:0/2607859614 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe7d001c450 con 0x7fe7dc106f40 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.644+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/2607859614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7dc194ce0 con 0x7fe7dc106f40 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.648+0000 7fe7d9ffb700 1 -- 172.21.15.7:0/2607859614 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe7d0011b90 con 0x7fe7dc106f40 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.722+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/2607859614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe7dc108240 con 0x7fe7dc106f40 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.723+0000 7fe7d9ffb700 1 -- 172.21.15.7:0/2607859614 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fe7d0011d30 con 0x7fe7dc106f40 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.724+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/2607859614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 msgr2=0x7fe7dc19b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.724+0000 7fe7e2ae8700 1 --2- 172.21.15.7:0/2607859614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc19b050 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0681b0 tx=0x7fe7d0004750 comp rx=0 tx=0).stop 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.724+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/2607859614 shutdown_connections 2024-03-23T13:09:40.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.724+0000 7fe7e2ae8700 1 --2- 172.21.15.7:0/2607859614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc106f40 0x7fe7dc19b050 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:40.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.724+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/2607859614 >> 172.21.15.7:0/2607859614 conn(0x7fe7dc1017a0 msgr2=0x7fe7dc191e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:40.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.724+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/2607859614 shutdown_connections 2024-03-23T13:09:40.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:39.724+0000 7fe7e2ae8700 1 -- 172.21.15.7:0/2607859614 wait complete. 2024-03-23T13:09:40.296 INFO:teuthology.orchestra.run.smithi007.stdout:mon is available 2024-03-23T13:09:40.296 INFO:teuthology.orchestra.run.smithi007.stdout:Assimilating anything we can from ceph.conf... 2024-03-23T13:09:41.566 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:41 smithi007 ceph-mon[31803]: from='client.? 172.21.15.7:0/1306694935' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-23T13:09:41.567 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:41 smithi007 ceph-mon[31803]: from='client.? 172.21.15.7:0/1306694935' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-23T13:09:44.824 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:09:44.824 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-03-23T13:09:44.824 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:09:44.824 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-03-23T13:09:44.824 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-23T13:09:44.824 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-23T13:09:44.824 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.700+0000 7fb181c86700 1 Processor -- start 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.700+0000 7fb181c86700 1 -- start start 2024-03-23T13:09:44.825 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.700+0000 7fb181c86700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c108000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.700+0000 7fb181c86700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1085d0 con 0x7fb17c107be0 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.700+0000 7fb17b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.700+0000 7fb17b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c108000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57960/0 (socket says 172.21.15.7:57960) 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.700+0000 7fb17b7fe700 1 -- 172.21.15.7:0/2006333932 learned_addr learned my addr 172.21.15.7:0/2006333932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb17b7fe700 1 -- 172.21.15.7:0/2006333932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c108d40 con 0x7fb17c107be0 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb17b7fe700 1 --2- 172.21.15.7:0/2006333932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c108000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0098a0 tx=0x7fb16c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6906ef92ed6e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb17a7fc700 1 -- 172.21.15.7:0/2006333932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb16c004160 con 0x7fb17c107be0 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb17a7fc700 1 -- 172.21.15.7:0/2006333932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16c0042c0 con 0x7fb17c107be0 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb17a7fc700 1 -- 172.21.15.7:0/2006333932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb16c0045a0 con 0x7fb17c107be0 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb181c86700 1 -- 172.21.15.7:0/2006333932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 msgr2=0x7fb17c108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb181c86700 1 --2- 172.21.15.7:0/2006333932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c108000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0098a0 tx=0x7fb16c009bb0 comp rx=0 tx=0).stop 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb181c86700 1 -- 172.21.15.7:0/2006333932 shutdown_connections 2024-03-23T13:09:44.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb181c86700 1 --2- 172.21.15.7:0/2006333932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c108000 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb181c86700 1 -- 172.21.15.7:0/2006333932 >> 172.21.15.7:0/2006333932 conn(0x7fb17c1031c0 msgr2=0x7fb17c1055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb181c86700 1 -- 172.21.15.7:0/2006333932 shutdown_connections 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.701+0000 7fb181c86700 1 -- 172.21.15.7:0/2006333932 wait complete. 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.702+0000 7fb181c86700 1 Processor -- start 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.702+0000 7fb181c86700 1 -- start start 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.702+0000 7fb181c86700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c19b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.702+0000 7fb181c86700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c19b5b0 con 0x7fb17c107be0 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.702+0000 7fb17b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c19b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.702+0000 7fb17b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c19b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57972/0 (socket says 172.21.15.7:57972) 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.702+0000 7fb17b7fe700 1 -- 172.21.15.7:0/1306694935 learned_addr learned my addr 172.21.15.7:0/1306694935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.703+0000 7fb17b7fe700 1 -- 172.21.15.7:0/1306694935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c009550 con 0x7fb17c107be0 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.703+0000 7fb17b7fe700 1 --2- 172.21.15.7:0/1306694935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c19b070 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb16c004000 tx=0x7fb16c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:44.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.703+0000 7fb178ff9700 1 -- 172.21.15.7:0/1306694935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb16c011630 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.703+0000 7fb178ff9700 1 -- 172.21.15.7:0/1306694935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16c011790 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.703+0000 7fb181c86700 1 -- 172.21.15.7:0/1306694935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17c19b7b0 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.703+0000 7fb178ff9700 1 -- 172.21.15.7:0/1306694935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb16c011a30 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.703+0000 7fb181c86700 1 -- 172.21.15.7:0/1306694935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17c19bc50 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.703+0000 7fb178ff9700 1 -- 172.21.15.7:0/1306694935 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb16c022070 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.703+0000 7fb178ff9700 1 -- 172.21.15.7:0/1306694935 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb16c01c450 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.704+0000 7fb181c86700 1 -- 172.21.15.7:0/1306694935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb17c194d60 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.706+0000 7fb178ff9700 1 -- 172.21.15.7:0/1306694935 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb16c011b90 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.777+0000 7fb181c86700 1 -- 172.21.15.7:0/1306694935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb17c109320 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.780+0000 7fb178ff9700 1 -- 172.21.15.7:0/1306694935 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7fb16c011de0 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.781+0000 7fb178ff9700 1 -- 172.21.15.7:0/1306694935 <== mon.0 v2:172.21.15.7:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb16c01c6b0 con 0x7fb17c107be0 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.781+0000 7fb181c86700 1 -- 172.21.15.7:0/1306694935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 msgr2=0x7fb17c19b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:44.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.781+0000 7fb181c86700 1 --2- 172.21.15.7:0/1306694935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c19b070 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb16c004000 tx=0x7fb16c004750 comp rx=0 tx=0).stop 2024-03-23T13:09:44.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.781+0000 7fb181c86700 1 -- 172.21.15.7:0/1306694935 shutdown_connections 2024-03-23T13:09:44.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.781+0000 7fb181c86700 1 --2- 172.21.15.7:0/1306694935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb17c107be0 0x7fb17c19b070 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:44.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.781+0000 7fb181c86700 1 -- 172.21.15.7:0/1306694935 >> 172.21.15.7:0/1306694935 conn(0x7fb17c1031c0 msgr2=0x7fb17c191ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:44.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.782+0000 7fb181c86700 1 -- 172.21.15.7:0/1306694935 shutdown_connections 2024-03-23T13:09:44.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:40.782+0000 7fb181c86700 1 -- 172.21.15.7:0/1306694935 wait complete. 2024-03-23T13:09:44.829 INFO:teuthology.orchestra.run.smithi007.stdout:Generating new minimal ceph.conf... 2024-03-23T13:09:48.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:48 smithi007 ceph-mon[31803]: from='client.? 172.21.15.7:0/3913512713' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:09:51.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.886+0000 7ff20032e700 1 Processor -- start 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.887+0000 7ff20032e700 1 -- start start 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.887+0000 7ff20032e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f810aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.887+0000 7ff20032e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f807be40 con 0x7ff1f8108630 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.887+0000 7ff1fe0ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f810aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.887+0000 7ff1fe0ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f810aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45660/0 (socket says 172.21.15.7:45660) 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.887+0000 7ff1fe0ca700 1 -- 172.21.15.7:0/2586106365 learned_addr learned my addr 172.21.15.7:0/2586106365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.887+0000 7ff1fe0ca700 1 -- 172.21.15.7:0/2586106365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f810af60 con 0x7ff1f8108630 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff1fe0ca700 1 --2- 172.21.15.7:0/2586106365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f810aa20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40098a0 tx=0x7ff1f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9476ab0623d3e590 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff1fd0c8700 1 -- 172.21.15.7:0/2586106365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f4004160 con 0x7ff1f8108630 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff1fd0c8700 1 -- 172.21.15.7:0/2586106365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff1f40042c0 con 0x7ff1f8108630 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff1fd0c8700 1 -- 172.21.15.7:0/2586106365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f4011ac0 con 0x7ff1f8108630 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff20032e700 1 -- 172.21.15.7:0/2586106365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 msgr2=0x7ff1f810aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:51.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff20032e700 1 --2- 172.21.15.7:0/2586106365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f810aa20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40098a0 tx=0x7ff1f4009bb0 comp rx=0 tx=0).stop 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff20032e700 1 -- 172.21.15.7:0/2586106365 shutdown_connections 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff20032e700 1 --2- 172.21.15.7:0/2586106365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f810aa20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff20032e700 1 -- 172.21.15.7:0/2586106365 >> 172.21.15.7:0/2586106365 conn(0x7ff1f8102040 msgr2=0x7ff1f8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff20032e700 1 -- 172.21.15.7:0/2586106365 shutdown_connections 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.888+0000 7ff20032e700 1 -- 172.21.15.7:0/2586106365 wait complete. 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.889+0000 7ff20032e700 1 Processor -- start 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.889+0000 7ff20032e700 1 -- start start 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.889+0000 7ff20032e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.889+0000 7ff20032e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f819be60 con 0x7ff1f8108630 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.889+0000 7ff1fe0ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.889+0000 7ff1fe0ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f819b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45676/0 (socket says 172.21.15.7:45676) 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.889+0000 7ff1fe0ca700 1 -- 172.21.15.7:0/3913512713 learned_addr learned my addr 172.21.15.7:0/3913512713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:51.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.890+0000 7ff1fe0ca700 1 -- 172.21.15.7:0/3913512713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f4009550 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.890+0000 7ff1fe0ca700 1 --2- 172.21.15.7:0/3913512713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f819b920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009130 tx=0x7ff1f4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.890+0000 7ff1ef7fe700 1 -- 172.21.15.7:0/3913512713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f4003aa0 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.890+0000 7ff1ef7fe700 1 -- 172.21.15.7:0/3913512713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff1f4003c00 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.890+0000 7ff1ef7fe700 1 -- 172.21.15.7:0/3913512713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f4024800 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.890+0000 7ff20032e700 1 -- 172.21.15.7:0/3913512713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f819c060 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.890+0000 7ff20032e700 1 -- 172.21.15.7:0/3913512713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f819c500 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.890+0000 7ff1ef7fe700 1 -- 172.21.15.7:0/3913512713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff1f402d3f0 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.890+0000 7ff1ef7fe700 1 -- 172.21.15.7:0/3913512713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff1f40197b0 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.891+0000 7ff20032e700 1 -- 172.21.15.7:0/3913512713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f810bd60 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.893+0000 7ff1ef7fe700 1 -- 172.21.15.7:0/3913512713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff1f4024e50 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.964+0000 7ff20032e700 1 -- 172.21.15.7:0/3913512713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff1f819f130 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.964+0000 7ff1ef7fe700 1 -- 172.21.15.7:0/3913512713 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7ff1f401abd0 con 0x7ff1f8108630 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.965+0000 7ff20032e700 1 -- 172.21.15.7:0/3913512713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 msgr2=0x7ff1f819b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:51.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.965+0000 7ff20032e700 1 --2- 172.21.15.7:0/3913512713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f819b920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009130 tx=0x7ff1f4011b40 comp rx=0 tx=0).stop 2024-03-23T13:09:51.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.965+0000 7ff20032e700 1 -- 172.21.15.7:0/3913512713 shutdown_connections 2024-03-23T13:09:51.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.965+0000 7ff20032e700 1 --2- 172.21.15.7:0/3913512713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1f8108630 0x7ff1f819b920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:51.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.965+0000 7ff20032e700 1 -- 172.21.15.7:0/3913512713 >> 172.21.15.7:0/3913512713 conn(0x7ff1f8102040 msgr2=0x7ff1f8102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:51.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.965+0000 7ff20032e700 1 -- 172.21.15.7:0/3913512713 shutdown_connections 2024-03-23T13:09:51.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:47.965+0000 7ff20032e700 1 -- 172.21.15.7:0/3913512713 wait complete. 2024-03-23T13:09:51.218 INFO:teuthology.orchestra.run.smithi007.stdout:Restarting the monitor... 2024-03-23T13:09:51.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:51 smithi007 systemd[1]: Stopping Ceph mon.a for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:09:51.809 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:51 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a[31798]: 2024-03-23T13:09:51.469+0000 7f0942c39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:09:51.809 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:51 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a[31798]: 2024-03-23T13:09:51.469+0000 7f0942c39700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-23T13:09:51.809 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:51 smithi007 podman[32058]: 2024-03-23 13:09:51.561663335 +0000 UTC m=+0.167837786 container died 0727ca53ac62f802d16c6bf4b558a442c9cb1ac0c8e145b1381f1518138f1e4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-23T13:09:52.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:51 smithi007 podman[32058]: 2024-03-23 13:09:51.936453598 +0000 UTC m=+0.542628045 container cleanup 0727ca53ac62f802d16c6bf4b558a442c9cb1ac0c8e145b1381f1518138f1e4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-03-23T13:09:52.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:51 smithi007 bash[32058]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a 2024-03-23T13:09:52.566 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 podman[32070]: 2024-03-23 13:09:52.303903088 +0000 UTC m=+0.743351525 container remove 0727ca53ac62f802d16c6bf4b558a442c9cb1ac0c8e145b1381f1518138f1e4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-23T13:09:52.567 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 systemd[1]: ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-23T13:09:52.567 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 systemd[1]: Stopped Ceph mon.a for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:09:52.567 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 systemd[1]: Starting Ceph mon.a for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:09:52.954 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 podman[32166]: 2024-03-23 13:09:52.704913437 +0000 UTC m=+0.091202439 container create d3f413d67b18980d577dbd7fb25ee5dacd8eb8b701a81b1ddfb2c69a27b4bc53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 podman[32166]: 2024-03-23 13:09:52.631302089 +0000 UTC m=+0.017591071 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 podman[32166]: 2024-03-23 13:09:52.882025474 +0000 UTC m=+0.268314446 container init d3f413d67b18980d577dbd7fb25ee5dacd8eb8b701a81b1ddfb2c69a27b4bc53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 podman[32166]: 2024-03-23 13:09:52.884935407 +0000 UTC m=+0.271224379 container start d3f413d67b18980d577dbd7fb25ee5dacd8eb8b701a81b1ddfb2c69a27b4bc53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev), process ceph-mon, pid 2 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: pidfile_write: ignore empty --pid-file 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: load: jerasure load: lrc 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: RocksDB version: 7.9.2 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Git sha 0 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Compile date 2024-03-23 03:28:38 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: DB SUMMARY 2024-03-23T13:09:52.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: DB Session ID: XAKBASRGIKFPLV5TQY0Z 2024-03-23T13:09:52.956 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: CURRENT file: CURRENT 2024-03-23T13:09:52.956 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T13:09:52.956 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-23T13:09:52.956 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-23T13:09:52.956 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104254 ; 2024-03-23T13:09:52.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.error_if_exists: 0 2024-03-23T13:09:52.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.create_if_missing: 0 2024-03-23T13:09:52.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.paranoid_checks: 1 2024-03-23T13:09:52.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-23T13:09:52.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T13:09:52.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-23T13:09:52.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.env: 0x55742f7d7c20 2024-03-23T13:09:52.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.fs: PosixFileSystem 2024-03-23T13:09:52.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.info_log: 0x557431261ac0 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.statistics: (nil) 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.use_fsync: 0 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_log_file_size: 0 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.allow_fallocate: 1 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T13:09:52.958 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.use_direct_reads: 0 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.db_log_dir: 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.wal_dir: 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T13:09:52.959 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.write_buffer_manager: 0x5574312341e0 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.unordered_write: 0 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.row_cache: None 2024-03-23T13:09:52.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.wal_filter: None 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.two_write_queues: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.wal_compression: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.atomic_flush: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.log_readahead_size: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T13:09:52.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_background_jobs: 2 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_background_compactions: -1 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_subcompactions: 1 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T13:09:52.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_open_files: -1 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_background_flushes: -1 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Compression algorithms supported: 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: kZSTD supported: 0 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: kXpressCompression supported: 0 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: kZlibCompression supported: 1 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: kSnappyCompression supported: 1 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: kLZ4Compression supported: 1 2024-03-23T13:09:52.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: kBZip2Compression supported: 0 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.merge_operator: 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_filter: None 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557431261bc0) 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-03-23T13:09:52.964 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_type: 0 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_index_type: 0 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_shortening: 1 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: checksum: 4 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: no_block_cache: 0 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache: 0x557431265090 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_options: 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: capacity : 536870912 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_shard_bits : 4 2024-03-23T13:09:52.965 INFO:journalctl@ceph.mon.a.smithi007.stdout: strict_capacity_limit : 0 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_compressed: (nil) 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: persistent_cache: (nil) 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size: 4096 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size_deviation: 10 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_restart_interval: 16 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_block_restart_interval: 1 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: metadata_block_size: 4096 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: partition_filters: 0 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: use_delta_encoding: 1 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: filter_policy: bloomfilter 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: whole_key_filtering: 1 2024-03-23T13:09:52.966 INFO:journalctl@ceph.mon.a.smithi007.stdout: verify_compression: 0 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout: format_version: 5 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_index_compression: 1 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_align: 0 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout: max_auto_readahead_size: 262144 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout: prepopulate_block_cache: 0 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout: initial_auto_readahead_size: 8192 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compression: NoCompression 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T13:09:52.967 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.num_levels: 7 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-23T13:09:52.968 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T13:09:52.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T13:09:52.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T13:09:52.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T13:09:52.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T13:09:52.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-23T13:09:52.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T13:09:52.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T13:09:52.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-23T13:09:52.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T13:09:52.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T13:09:52.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T13:09:52.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T13:09:52.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T13:09:52.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T13:09:52.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T13:09:52.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.arena_block_size: 1048576 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T13:09:52.971 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.table_properties_collectors: 2024-03-23T13:09:52.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.inplace_update_support: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.bloom_locality: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.max_successive_merges: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.ttl: 2592000 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-23T13:09:52.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.enable_blob_files: false 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.min_blob_size: 0 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.blob_file_starting_level: 0 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a1820fe8-174e-4cd4-9d6d-6b7d4d055161 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711199392913994, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-23T13:09:52.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711199392915010, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99413, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97321, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14054, "raw_average_key_size": 49, "raw_value_size": 89695, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711199392, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a1820fe8-174e-4cd4-9d6d-6b7d4d055161", "db_session_id": "XAKBASRGIKFPLV5TQY0Z", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711199392915093, "job": 1, "event": "recovery_finished"} 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557431356000 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: DB pointer 0x557431340000 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** DB Stats ** 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T13:09:52.975 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: L0 2/0 98.92 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: Sum 2/0 98.92 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T13:09:52.976 INFO:journalctl@ceph.mon.a.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative compaction: 0.00 GB write, 31.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval compaction: 0.00 GB write, 31.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache BinnedLRUCache@0x557431265090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-23T13:09:52.977 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: starting mon.a rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: mon.a@-1(???) e1 preinit fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: mon.a@-1(???).mds e1 new map 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: mon.a@-1(???).mds e1 print_map 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout: e1 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout: legacy client fscid: -1 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout: No filesystems configured 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T13:09:52.978 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T13:09:52.979 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T13:09:52.979 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-23T13:09:52.979 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 bash[32166]: d3f413d67b18980d577dbd7fb25ee5dacd8eb8b701a81b1ddfb2c69a27b4bc53 2024-03-23T13:09:52.979 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 systemd[1]: Started Ceph mon.a for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:09:52.979 INFO:teuthology.orchestra.run.smithi007.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: monmap epoch 1 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: last_changed 2024-03-23T13:09:36.205070+0000 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: created 2024-03-23T13:09:36.205070+0000 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: min_mon_release 19 (squid) 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: election_strategy: 1 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: fsmap 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: osdmap e1: 0 total, 0 up, 0 in 2024-03-23T13:09:53.224 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:52 smithi007 ceph-mon[32184]: mgrmap e1: no daemons active 2024-03-23T13:09:53.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.337+0000 7fcc2caff700 1 Processor -- start 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.337+0000 7fcc2caff700 1 -- start start 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.337+0000 7fcc2caff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.337+0000 7fcc2caff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2807c830 con 0x7fcc2807ddf0 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.337+0000 7fcc2659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.337+0000 7fcc2659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45690/0 (socket says 172.21.15.7:45690) 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.337+0000 7fcc2659c700 1 -- 172.21.15.7:0/1444250065 learned_addr learned my addr 172.21.15.7:0/1444250065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.338+0000 7fcc2659c700 1 -- 172.21.15.7:0/1444250065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2807c970 con 0x7fcc2807ddf0 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.338+0000 7fcc2659c700 1 --2- 172.21.15.7:0/1444250065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2807c2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcc180098a0 tx=0x7fcc18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea236733cc3faaf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.338+0000 7fcc2559a700 1 -- 172.21.15.7:0/1444250065 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc18004160 con 0x7fcc2807ddf0 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.338+0000 7fcc2559a700 1 -- 172.21.15.7:0/1444250065 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc180042c0 con 0x7fcc2807ddf0 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.338+0000 7fcc2559a700 1 -- 172.21.15.7:0/1444250065 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc18011ac0 con 0x7fcc2807ddf0 2024-03-23T13:09:53.968 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.339+0000 7fcc2caff700 1 -- 172.21.15.7:0/1444250065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 msgr2=0x7fcc2807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.339+0000 7fcc2caff700 1 --2- 172.21.15.7:0/1444250065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2807c2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcc180098a0 tx=0x7fcc18009bb0 comp rx=0 tx=0).stop 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.339+0000 7fcc2caff700 1 -- 172.21.15.7:0/1444250065 shutdown_connections 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.339+0000 7fcc2caff700 1 --2- 172.21.15.7:0/1444250065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2807c2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.339+0000 7fcc2caff700 1 -- 172.21.15.7:0/1444250065 >> 172.21.15.7:0/1444250065 conn(0x7fcc28103200 msgr2=0x7fcc28105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.339+0000 7fcc2caff700 1 -- 172.21.15.7:0/1444250065 shutdown_connections 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.339+0000 7fcc2caff700 1 -- 172.21.15.7:0/1444250065 wait complete. 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.339+0000 7fcc2caff700 1 Processor -- start 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.339+0000 7fcc2caff700 1 -- start start 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc2caff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc2caff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2819e000 con 0x7fcc2807ddf0 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc2659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc2659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45698/0 (socket says 172.21.15.7:45698) 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc2659c700 1 -- 172.21.15.7:0/1144662832 learned_addr learned my addr 172.21.15.7:0/1144662832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc2659c700 1 -- 172.21.15.7:0/1144662832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc18009550 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc2659c700 1 --2- 172.21.15.7:0/1144662832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2819dac0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc18003f70 tx=0x7fcc18011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc177fe700 1 -- 172.21.15.7:0/1144662832 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc180039f0 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc177fe700 1 -- 172.21.15.7:0/1144662832 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc18003b50 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc2caff700 1 -- 172.21.15.7:0/1144662832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2819e200 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc177fe700 1 -- 172.21.15.7:0/1144662832 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc1801a760 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.340+0000 7fcc2caff700 1 -- 172.21.15.7:0/1144662832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2819e6a0 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.341+0000 7fcc177fe700 1 -- 172.21.15.7:0/1144662832 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc1801ac90 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.341+0000 7fcc177fe700 1 -- 172.21.15.7:0/1144662832 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc180199b0 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.341+0000 7fcc2caff700 1 -- 172.21.15.7:0/1144662832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc2810df50 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.344+0000 7fcc177fe700 1 -- 172.21.15.7:0/1144662832 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcc180249f0 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.414+0000 7fcc2caff700 1 -- 172.21.15.7:0/1144662832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fcc2810e160 con 0x7fcc2807ddf0 2024-03-23T13:09:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.415+0000 7fcc177fe700 1 -- 172.21.15.7:0/1144662832 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fcc18019b70 con 0x7fcc2807ddf0 2024-03-23T13:09:53.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.416+0000 7fcc2caff700 1 -- 172.21.15.7:0/1144662832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 msgr2=0x7fcc2819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:53.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.416+0000 7fcc2caff700 1 --2- 172.21.15.7:0/1144662832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2819dac0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc18003f70 tx=0x7fcc18011b40 comp rx=0 tx=0).stop 2024-03-23T13:09:53.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.416+0000 7fcc2caff700 1 -- 172.21.15.7:0/1144662832 shutdown_connections 2024-03-23T13:09:53.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.416+0000 7fcc2caff700 1 --2- 172.21.15.7:0/1144662832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2807ddf0 0x7fcc2819dac0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:53.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.416+0000 7fcc2caff700 1 -- 172.21.15.7:0/1144662832 >> 172.21.15.7:0/1144662832 conn(0x7fcc28103200 msgr2=0x7fcc28103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:53.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.416+0000 7fcc2caff700 1 -- 172.21.15.7:0/1144662832 shutdown_connections 2024-03-23T13:09:53.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:53.416+0000 7fcc2caff700 1 -- 172.21.15.7:0/1144662832 wait complete. 2024-03-23T13:09:53.971 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-23T13:09:53.993 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-23T13:09:53.993 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mgr... 2024-03-23T13:09:53.994 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-23T13:09:53.994 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-23T13:09:54.241 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.x 2024-03-23T13:09:54.241 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-23T13:09:54.390 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-745f591a-e916-11ee-95cd-87774f69a715.target.wants/ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-745f591a-e916-11ee-95cd-87774f69a715@.service. 2024-03-23T13:09:54.567 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:54 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1144662832' entity='client.admin' 2024-03-23T13:09:54.567 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:54 smithi007 systemd[1]: Starting Ceph mgr.x for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:09:54.882 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:54 smithi007 podman[32400]: 2024-03-23 13:09:54.53411733 +0000 UTC m=+0.017186431 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T13:09:54.882 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:54 smithi007 podman[32400]: 2024-03-23 13:09:54.6317849 +0000 UTC m=+0.114853990 container create 67d150695aee77b534d96595087965c6e69f62c5976b69a1dddac7a3c4a1d72f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-03-23T13:09:54.882 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:54 smithi007 podman[32400]: 2024-03-23 13:09:54.842383628 +0000 UTC m=+0.325452734 container init 67d150695aee77b534d96595087965c6e69f62c5976b69a1dddac7a3c4a1d72f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-23T13:09:54.882 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:54 smithi007 podman[32400]: 2024-03-23 13:09:54.8448272 +0000 UTC m=+0.327896293 container start 67d150695aee77b534d96595087965c6e69f62c5976b69a1dddac7a3c4a1d72f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-23T13:09:54.882 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:54 smithi007 bash[32400]: 67d150695aee77b534d96595087965c6e69f62c5976b69a1dddac7a3c4a1d72f 2024-03-23T13:09:54.897 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T13:09:54.897 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-23T13:09:54.905 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T13:09:54.905 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-23T13:09:54.905 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-23T13:09:54.905 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-03-23T13:09:54.915 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T13:09:54.915 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-23T13:09:54.924 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T13:09:54.924 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-23T13:09:54.924 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-23T13:09:54.924 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-23T13:09:54.924 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr to start... 2024-03-23T13:09:54.924 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr... 2024-03-23T13:09:55.166 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:54 smithi007 systemd[1]: Started Ceph mgr.x for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:09:55.817 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:55 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1194413782' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T13:09:55.817 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:55 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:55.427+0000 7f1c3b47d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T13:09:55.817 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:55 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:55.531+0000 7f1c3b47d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T13:09:56.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:09:56.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-23T13:09:56.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:09:56.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T13:09:56.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T13:09:56.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T13:09:56.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:56.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T13:09:56.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:09:56.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T13:09:56.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T13:09:56.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T13:09:56.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T13:09:56.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T13:09:56.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T13:09:56.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:56.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T13:09:56.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T13:09:56.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T13:09:56.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T13:09:56.311 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T13:09:39.214309+0000", 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.372+0000 7fb41c98f700 1 Processor -- start 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.372+0000 7fb41c98f700 1 -- start start 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.372+0000 7fb41c98f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb418107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:56.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.372+0000 7fb41c98f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb418108590 con 0x7fb418107ba0 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.372+0000 7fb41659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb418107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.372+0000 7fb41659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb418107fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59116/0 (socket says 172.21.15.7:59116) 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.372+0000 7fb41659c700 1 -- 172.21.15.7:0/3670882584 learned_addr learned my addr 172.21.15.7:0/3670882584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.373+0000 7fb41659c700 1 -- 172.21.15.7:0/3670882584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb418108d50 con 0x7fb418107ba0 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.373+0000 7fb41659c700 1 --2- 172.21.15.7:0/3670882584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb418107fc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb4080098a0 tx=0x7fb408009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6cf6b299c06bbf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.373+0000 7fb41559a700 1 -- 172.21.15.7:0/3670882584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb408004160 con 0x7fb418107ba0 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.373+0000 7fb41559a700 1 -- 172.21.15.7:0/3670882584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4080042c0 con 0x7fb418107ba0 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.373+0000 7fb41559a700 1 -- 172.21.15.7:0/3670882584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb408011ac0 con 0x7fb418107ba0 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.374+0000 7fb41c98f700 1 -- 172.21.15.7:0/3670882584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 msgr2=0x7fb418107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.374+0000 7fb41c98f700 1 --2- 172.21.15.7:0/3670882584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb418107fc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb4080098a0 tx=0x7fb408009bb0 comp rx=0 tx=0).stop 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.374+0000 7fb41c98f700 1 -- 172.21.15.7:0/3670882584 shutdown_connections 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.374+0000 7fb41c98f700 1 --2- 172.21.15.7:0/3670882584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb418107fc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.374+0000 7fb41c98f700 1 -- 172.21.15.7:0/3670882584 >> 172.21.15.7:0/3670882584 conn(0x7fb418103160 msgr2=0x7fb418105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.374+0000 7fb41c98f700 1 -- 172.21.15.7:0/3670882584 shutdown_connections 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.374+0000 7fb41c98f700 1 -- 172.21.15.7:0/3670882584 wait complete. 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb41c98f700 1 Processor -- start 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb41c98f700 1 -- start start 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb41c98f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb41819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb41c98f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb41819bed0 con 0x7fb418107ba0 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb41659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb41819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb41659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb41819b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59128/0 (socket says 172.21.15.7:59128) 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb41659c700 1 -- 172.21.15.7:0/1194413782 learned_addr learned my addr 172.21.15.7:0/1194413782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb41659c700 1 -- 172.21.15.7:0/1194413782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb408009550 con 0x7fb418107ba0 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb41659c700 1 --2- 172.21.15.7:0/1194413782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb41819b990 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb408003f70 tx=0x7fb408011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb40f7fe700 1 -- 172.21.15.7:0/1194413782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4080039f0 con 0x7fb418107ba0 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb41c98f700 1 -- 172.21.15.7:0/1194413782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41819c0d0 con 0x7fb418107ba0 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.375+0000 7fb40f7fe700 1 -- 172.21.15.7:0/1194413782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb408003b50 con 0x7fb418107ba0 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.376+0000 7fb40f7fe700 1 -- 172.21.15.7:0/1194413782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb40801a690 con 0x7fb418107ba0 2024-03-23T13:09:56.314 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.376+0000 7fb41c98f700 1 -- 172.21.15.7:0/1194413782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41819c570 con 0x7fb418107ba0 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.376+0000 7fb40f7fe700 1 -- 172.21.15.7:0/1194413782 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb40801ad00 con 0x7fb418107ba0 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.376+0000 7fb40f7fe700 1 -- 172.21.15.7:0/1194413782 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb4080199c0 con 0x7fb418107ba0 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.376+0000 7fb41c98f700 1 -- 172.21.15.7:0/1194413782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41810bf80 con 0x7fb418107ba0 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.380+0000 7fb40f7fe700 1 -- 172.21.15.7:0/1194413782 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb4080249f0 con 0x7fb418107ba0 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.454+0000 7fb41c98f700 1 -- 172.21.15.7:0/1194413782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb41810c190 con 0x7fb418107ba0 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.455+0000 7fb40f7fe700 1 -- 172.21.15.7:0/1194413782 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb408019b80 con 0x7fb418107ba0 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.456+0000 7fb41c98f700 1 -- 172.21.15.7:0/1194413782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 msgr2=0x7fb41819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.456+0000 7fb41c98f700 1 --2- 172.21.15.7:0/1194413782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb41819b990 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb408003f70 tx=0x7fb408011b40 comp rx=0 tx=0).stop 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.456+0000 7fb41c98f700 1 -- 172.21.15.7:0/1194413782 shutdown_connections 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.456+0000 7fb41c98f700 1 --2- 172.21.15.7:0/1194413782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb418107ba0 0x7fb41819b990 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.456+0000 7fb41c98f700 1 -- 172.21.15.7:0/1194413782 >> 172.21.15.7:0/1194413782 conn(0x7fb418103160 msgr2=0x7fb418103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.456+0000 7fb41c98f700 1 -- 172.21.15.7:0/1194413782 shutdown_connections 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:55.456+0000 7fb41c98f700 1 -- 172.21.15.7:0/1194413782 wait complete. 2024-03-23T13:09:56.315 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (1/15)... 2024-03-23T13:09:56.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:56 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:56.199+0000 7f1c3b47d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T13:09:56.534 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:56 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:56.294+0000 7f1c3b47d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T13:09:56.534 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:56 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:56.439+0000 7f1c3b47d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T13:09:56.534 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:56 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:56.544+0000 7f1c3b47d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T13:09:56.817 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:56 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:56.630+0000 7f1c3b47d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T13:09:57.190 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:56 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:56.903+0000 7f1c3b47d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T13:09:57.190 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:57 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:57.015+0000 7f1c3b47d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T13:09:57.190 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:57 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:57.103+0000 7f1c3b47d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T13:09:57.190 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:57 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:57.200+0000 7f1c3b47d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T13:09:57.462 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:57 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:57.380+0000 7f1c3b47d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T13:09:58.713 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:58 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:58.401+0000 7f1c3b47d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T13:09:58.713 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:58 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:58.539+0000 7f1c3b47d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T13:09:58.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:09:58 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2975456085' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T13:09:58.977 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:58 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:58.893+0000 7f1c3b47d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T13:09:58.977 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:58 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:58.986+0000 7f1c3b47d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T13:09:59.317 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:59 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:59.190+0000 7f1c3b47d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T13:09:59.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:09:59.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T13:09:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T13:09:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T13:09:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T13:09:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T13:09:39.214309+0000", 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.751+0000 7f44f46f6700 1 Processor -- start 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.751+0000 7f44f46f6700 1 -- start start 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.751+0000 7f44f46f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.751+0000 7f44f46f6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1089e0 con 0x7f44ec107ff0 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.752+0000 7f44f2492700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.752+0000 7f44f2492700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec108410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59138/0 (socket says 172.21.15.7:59138) 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.752+0000 7f44f2492700 1 -- 172.21.15.7:0/3700391108 learned_addr learned my addr 172.21.15.7:0/3700391108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.752+0000 7f44f2492700 1 -- 172.21.15.7:0/3700391108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec106c50 con 0x7f44ec107ff0 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.752+0000 7f44f2492700 1 --2- 172.21.15.7:0/3700391108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec108410 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f44e80098a0 tx=0x7f44e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5255803b8de8c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.752+0000 7f44f1490700 1 -- 172.21.15.7:0/3700391108 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44e8004160 con 0x7f44ec107ff0 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.753+0000 7f44f1490700 1 -- 172.21.15.7:0/3700391108 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f44e80042c0 con 0x7f44ec107ff0 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.753+0000 7f44f1490700 1 -- 172.21.15.7:0/3700391108 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44e8011ac0 con 0x7f44ec107ff0 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.753+0000 7f44f46f6700 1 -- 172.21.15.7:0/3700391108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 msgr2=0x7f44ec108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.753+0000 7f44f46f6700 1 --2- 172.21.15.7:0/3700391108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec108410 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f44e80098a0 tx=0x7f44e8009bb0 comp rx=0 tx=0).stop 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.754+0000 7f44f46f6700 1 -- 172.21.15.7:0/3700391108 shutdown_connections 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.754+0000 7f44f46f6700 1 --2- 172.21.15.7:0/3700391108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec108410 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.754+0000 7f44f46f6700 1 -- 172.21.15.7:0/3700391108 >> 172.21.15.7:0/3700391108 conn(0x7f44ec1017a0 msgr2=0x7f44ec103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.754+0000 7f44f46f6700 1 -- 172.21.15.7:0/3700391108 shutdown_connections 2024-03-23T13:09:59.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.754+0000 7f44f46f6700 1 -- 172.21.15.7:0/3700391108 wait complete. 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.754+0000 7f44f46f6700 1 Processor -- start 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44f46f6700 1 -- start start 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44f46f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec1065e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44f46f6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec19e620 con 0x7f44ec107ff0 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44f2492700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec1065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44f2492700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec1065e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59140/0 (socket says 172.21.15.7:59140) 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44f2492700 1 -- 172.21.15.7:0/2975456085 learned_addr learned my addr 172.21.15.7:0/2975456085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44f2492700 1 -- 172.21.15.7:0/2975456085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e8009550 con 0x7f44ec107ff0 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44f2492700 1 --2- 172.21.15.7:0/2975456085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec1065e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44e8003f70 tx=0x7f44e8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44e37fe700 1 -- 172.21.15.7:0/2975456085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44e80039f0 con 0x7f44ec107ff0 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44f46f6700 1 -- 172.21.15.7:0/2975456085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec19e760 con 0x7f44ec107ff0 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44e37fe700 1 -- 172.21.15.7:0/2975456085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f44e8003b50 con 0x7f44ec107ff0 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.755+0000 7f44e37fe700 1 -- 172.21.15.7:0/2975456085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44e801a760 con 0x7f44ec107ff0 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.756+0000 7f44f46f6700 1 -- 172.21.15.7:0/2975456085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec19eb40 con 0x7f44ec107ff0 2024-03-23T13:09:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.756+0000 7f44e37fe700 1 -- 172.21.15.7:0/2975456085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f44e801ac90 con 0x7f44ec107ff0 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.756+0000 7f44e37fe700 1 -- 172.21.15.7:0/2975456085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f44e80199b0 con 0x7f44ec107ff0 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.757+0000 7f44f46f6700 1 -- 172.21.15.7:0/2975456085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ec10ae90 con 0x7f44ec107ff0 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.759+0000 7f44e37fe700 1 -- 172.21.15.7:0/2975456085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f44e80249f0 con 0x7f44ec107ff0 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.833+0000 7f44f46f6700 1 -- 172.21.15.7:0/2975456085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f44ec10b0a0 con 0x7f44ec107ff0 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.834+0000 7f44e37fe700 1 -- 172.21.15.7:0/2975456085 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f44e8019c10 con 0x7f44ec107ff0 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.835+0000 7f44f46f6700 1 -- 172.21.15.7:0/2975456085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 msgr2=0x7f44ec1065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.835+0000 7f44f46f6700 1 --2- 172.21.15.7:0/2975456085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec1065e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44e8003f70 tx=0x7f44e8011b40 comp rx=0 tx=0).stop 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.835+0000 7f44f46f6700 1 -- 172.21.15.7:0/2975456085 shutdown_connections 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.835+0000 7f44f46f6700 1 --2- 172.21.15.7:0/2975456085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec107ff0 0x7f44ec1065e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.835+0000 7f44f46f6700 1 -- 172.21.15.7:0/2975456085 >> 172.21.15.7:0/2975456085 conn(0x7f44ec1017a0 msgr2=0x7f44ec1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.835+0000 7f44f46f6700 1 -- 172.21.15.7:0/2975456085 shutdown_connections 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:09:58.835+0000 7f44f46f6700 1 -- 172.21.15.7:0/2975456085 wait complete. 2024-03-23T13:09:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (2/15)... 2024-03-23T13:09:59.661 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:59 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:59.379+0000 7f1c3b47d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T13:09:59.921 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:59 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:59.671+0000 7f1c3b47d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T13:09:59.922 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:09:59 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:09:59.931+0000 7f1c3b47d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T13:10:00.317 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:00 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:00.175+0000 7f1c3b47d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T13:10:01.317 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:01.006+0000 7f1c3b47d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T13:10:01.817 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:01.588+0000 7f1c3b47d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T13:10:01.817 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:01.693+0000 7f1c3b47d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T13:10:02.317 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:01.925+0000 7f1c3b47d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T13:10:02.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: Activating manager daemon x 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: mgrmap e2: x(active, starting, since 0.00309522s) 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: from='mgr.14100 172.21.15.7:0/2186728198' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: from='mgr.14100 172.21.15.7:0/2186728198' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: from='mgr.14100 172.21.15.7:0/2186728198' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: from='mgr.14100 172.21.15.7:0/2186728198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: from='mgr.14100 172.21.15.7:0/2186728198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: Manager daemon x is now available 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: from='mgr.14100 172.21.15.7:0/2186728198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: from='mgr.14100 172.21.15.7:0/2186728198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: from='mgr.14100 172.21.15.7:0/2186728198' entity='mgr.x' 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: from='mgr.14100 172.21.15.7:0/2186728198' entity='mgr.x' 2024-03-23T13:10:02.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:01 smithi007 ceph-mon[32184]: from='mgr.14100 172.21.15.7:0/2186728198' entity='mgr.x' 2024-03-23T13:10:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:10:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T13:10:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T13:10:02.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T13:10:02.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T13:10:02.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T13:10:02.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:02.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T13:10:02.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T13:10:02.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T13:10:02.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T13:10:02.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T13:10:02.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T13:09:39.214309+0000", 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.913+0000 7f476bf95700 1 Processor -- start 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.913+0000 7f476bf95700 1 -- start start 2024-03-23T13:10:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.913+0000 7f476bf95700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476410aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.913+0000 7f476bf95700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f476407be40 con 0x7f4764108630 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.913+0000 7f4769d31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476410aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.913+0000 7f4769d31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476410aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59152/0 (socket says 172.21.15.7:59152) 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.913+0000 7f4769d31700 1 -- 172.21.15.7:0/3920912737 learned_addr learned my addr 172.21.15.7:0/3920912737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.914+0000 7f4769d31700 1 -- 172.21.15.7:0/3920912737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476410af60 con 0x7f4764108630 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.914+0000 7f4769d31700 1 --2- 172.21.15.7:0/3920912737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476410aa20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f47600098a0 tx=0x7f4760009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e10f4bbbe56cdd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.914+0000 7f4768d2f700 1 -- 172.21.15.7:0/3920912737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760004160 con 0x7f4764108630 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.914+0000 7f4768d2f700 1 -- 172.21.15.7:0/3920912737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f47600042c0 con 0x7f4764108630 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.914+0000 7f4768d2f700 1 -- 172.21.15.7:0/3920912737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760011ac0 con 0x7f4764108630 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.915+0000 7f476bf95700 1 -- 172.21.15.7:0/3920912737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 msgr2=0x7f476410aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.915+0000 7f476bf95700 1 --2- 172.21.15.7:0/3920912737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476410aa20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f47600098a0 tx=0x7f4760009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.915+0000 7f476bf95700 1 -- 172.21.15.7:0/3920912737 shutdown_connections 2024-03-23T13:10:02.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.915+0000 7f476bf95700 1 --2- 172.21.15.7:0/3920912737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476410aa20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.915+0000 7f476bf95700 1 -- 172.21.15.7:0/3920912737 >> 172.21.15.7:0/3920912737 conn(0x7f4764102040 msgr2=0x7f4764104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.915+0000 7f476bf95700 1 -- 172.21.15.7:0/3920912737 shutdown_connections 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.915+0000 7f476bf95700 1 -- 172.21.15.7:0/3920912737 wait complete. 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.916+0000 7f476bf95700 1 Processor -- start 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.916+0000 7f476bf95700 1 -- start start 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.916+0000 7f476bf95700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.916+0000 7f476bf95700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f476419be60 con 0x7f4764108630 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.916+0000 7f4769d31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.916+0000 7f4769d31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476419b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59162/0 (socket says 172.21.15.7:59162) 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.916+0000 7f4769d31700 1 -- 172.21.15.7:0/775284701 learned_addr learned my addr 172.21.15.7:0/775284701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.916+0000 7f4769d31700 1 -- 172.21.15.7:0/775284701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4760009550 con 0x7f4764108630 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.916+0000 7f4769d31700 1 --2- 172.21.15.7:0/775284701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476419b920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4760009130 tx=0x7f4760011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.917+0000 7f475affd700 1 -- 172.21.15.7:0/775284701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760003aa0 con 0x7f4764108630 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.917+0000 7f475affd700 1 -- 172.21.15.7:0/775284701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4760003c00 con 0x7f4764108630 2024-03-23T13:10:02.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.917+0000 7f476bf95700 1 -- 172.21.15.7:0/775284701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476419c060 con 0x7f4764108630 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.917+0000 7f475affd700 1 -- 172.21.15.7:0/775284701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760024800 con 0x7f4764108630 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.917+0000 7f475affd700 1 -- 172.21.15.7:0/775284701 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f47600193c0 con 0x7f4764108630 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.917+0000 7f476bf95700 1 -- 172.21.15.7:0/775284701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476419c500 con 0x7f4764108630 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.918+0000 7f475affd700 1 -- 172.21.15.7:0/775284701 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4760024e20 con 0x7f4764108630 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.918+0000 7f476bf95700 1 -- 172.21.15.7:0/775284701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476410bd60 con 0x7f4764108630 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.920+0000 7f475affd700 1 -- 172.21.15.7:0/775284701 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f476000b3e0 con 0x7f4764108630 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.940+0000 7f475affd700 1 -- 172.21.15.7:0/775284701 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 2) ==== 45684+0+0 (secure 0 0 0) 0x7f476001abd0 con 0x7f4764108630 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.998+0000 7f476bf95700 1 -- 172.21.15.7:0/775284701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f476410bf70 con 0x7f4764108630 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.998+0000 7f475affd700 1 -- 172.21.15.7:0/775284701 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f476000b8c0 con 0x7f4764108630 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.999+0000 7f476bf95700 1 -- 172.21.15.7:0/775284701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 msgr2=0x7f476419b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.999+0000 7f476bf95700 1 --2- 172.21.15.7:0/775284701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476419b920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4760009130 tx=0x7f4760011b40 comp rx=0 tx=0).stop 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.999+0000 7f476bf95700 1 -- 172.21.15.7:0/775284701 shutdown_connections 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.999+0000 7f476bf95700 1 --2- 172.21.15.7:0/775284701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764108630 0x7f476419b920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.999+0000 7f476bf95700 1 -- 172.21.15.7:0/775284701 >> 172.21.15.7:0/775284701 conn(0x7f4764102040 msgr2=0x7f4764102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:02.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:01.999+0000 7f476bf95700 1 -- 172.21.15.7:0/775284701 shutdown_connections 2024-03-23T13:10:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:02.000+0000 7f476bf95700 1 -- 172.21.15.7:0/775284701 wait complete. 2024-03-23T13:10:02.861 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (3/15)... 2024-03-23T13:10:03.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:02 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/775284701' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T13:10:03.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:02 smithi007 ceph-mon[32184]: mgrmap e3: x(active, since 1.00686s) 2024-03-23T13:10:05.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:04 smithi007 ceph-mon[32184]: mgrmap e4: x(active, since 2s) 2024-03-23T13:10:06.185 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:10:06.185 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-23T13:10:06.185 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:10:06.185 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T13:10:06.185 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T13:10:06.185 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T13:10:06.185 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T13:10:06.185 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:06.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T13:10:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T13:10:06.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T13:09:39.214309+0000", 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-23T13:10:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.340+0000 7f58d264a700 1 Processor -- start 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.340+0000 7f58d264a700 1 -- start start 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.340+0000 7f58d264a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc106e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.340+0000 7f58d264a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc107450 con 0x7f58cc106a60 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.340+0000 7f58cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc106e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.340+0000 7f58cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc106e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46478/0 (socket says 172.21.15.7:46478) 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.340+0000 7f58cbfff700 1 -- 172.21.15.7:0/1813726757 learned_addr learned my addr 172.21.15.7:0/1813726757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.341+0000 7f58cbfff700 1 -- 172.21.15.7:0/1813726757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc107c60 con 0x7f58cc106a60 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.341+0000 7f58cbfff700 1 --2- 172.21.15.7:0/1813726757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc106e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0098a0 tx=0x7f58bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664480531d794223 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.341+0000 7f58caffd700 1 -- 172.21.15.7:0/1813726757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58bc004160 con 0x7f58cc106a60 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.341+0000 7f58caffd700 1 -- 172.21.15.7:0/1813726757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58bc0042c0 con 0x7f58cc106a60 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.341+0000 7f58caffd700 1 -- 172.21.15.7:0/1813726757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58bc011ac0 con 0x7f58cc106a60 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.342+0000 7f58d264a700 1 -- 172.21.15.7:0/1813726757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 msgr2=0x7f58cc106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.342+0000 7f58d264a700 1 --2- 172.21.15.7:0/1813726757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc106e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0098a0 tx=0x7f58bc009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.342+0000 7f58d264a700 1 -- 172.21.15.7:0/1813726757 shutdown_connections 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.342+0000 7f58d264a700 1 --2- 172.21.15.7:0/1813726757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc106e80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.342+0000 7f58d264a700 1 -- 172.21.15.7:0/1813726757 >> 172.21.15.7:0/1813726757 conn(0x7f58cc102040 msgr2=0x7f58cc104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.342+0000 7f58d264a700 1 -- 172.21.15.7:0/1813726757 shutdown_connections 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.342+0000 7f58d264a700 1 -- 172.21.15.7:0/1813726757 wait complete. 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.342+0000 7f58d264a700 1 Processor -- start 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.342+0000 7f58d264a700 1 -- start start 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.343+0000 7f58d264a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc191bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.343+0000 7f58d264a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc192110 con 0x7f58cc106a60 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.343+0000 7f58cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc191bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.343+0000 7f58cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc191bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46490/0 (socket says 172.21.15.7:46490) 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.343+0000 7f58cbfff700 1 -- 172.21.15.7:0/564053559 learned_addr learned my addr 172.21.15.7:0/564053559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:06.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.343+0000 7f58cbfff700 1 -- 172.21.15.7:0/564053559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc009550 con 0x7f58cc106a60 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.343+0000 7f58cbfff700 1 --2- 172.21.15.7:0/564053559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc191bd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f58bc003f70 tx=0x7f58bc011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.343+0000 7f58c97fa700 1 -- 172.21.15.7:0/564053559 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58bc0039f0 con 0x7f58cc106a60 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.343+0000 7f58c97fa700 1 -- 172.21.15.7:0/564053559 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58bc003b50 con 0x7f58cc106a60 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.344+0000 7f58d264a700 1 -- 172.21.15.7:0/564053559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc190300 con 0x7f58cc106a60 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.344+0000 7f58c97fa700 1 -- 172.21.15.7:0/564053559 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58bc01a690 con 0x7f58cc106a60 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.344+0000 7f58d264a700 1 -- 172.21.15.7:0/564053559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc1907a0 con 0x7f58cc106a60 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.344+0000 7f58c97fa700 1 -- 172.21.15.7:0/564053559 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 45917+0+0 (secure 0 0 0) 0x7f58bc01a920 con 0x7f58cc106a60 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.344+0000 7f58d264a700 1 -- 172.21.15.7:0/564053559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc10ae90 con 0x7f58cc106a60 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.345+0000 7f58c97fa700 1 --2- 172.21.15.7:0/564053559 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f58b4038e90 0x7f58b403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.345+0000 7f58c97fa700 1 -- 172.21.15.7:0/564053559 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f58bc04d090 con 0x7f58cc106a60 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.345+0000 7f58cb7fe700 1 --2- 172.21.15.7:0/564053559 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f58b4038e90 0x7f58b403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.345+0000 7f58cb7fe700 1 --2- 172.21.15.7:0/564053559 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f58b4038e90 0x7f58b403b350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f58c0006fd0 tx=0x7f58c0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.350+0000 7f58c97fa700 1 -- 172.21.15.7:0/564053559 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f58bc01abd0 con 0x7f58cc106a60 2024-03-23T13:10:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.643+0000 7f58d264a700 1 -- 172.21.15.7:0/564053559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f58cc195750 con 0x7f58cc106a60 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.644+0000 7f58c97fa700 1 -- 172.21.15.7:0/564053559 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f58bc018350 con 0x7f58cc106a60 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.646+0000 7f58d264a700 1 -- 172.21.15.7:0/564053559 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f58b4038e90 msgr2=0x7f58b403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.646+0000 7f58d264a700 1 --2- 172.21.15.7:0/564053559 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f58b4038e90 0x7f58b403b350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f58c0006fd0 tx=0x7f58c0006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.646+0000 7f58d264a700 1 -- 172.21.15.7:0/564053559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 msgr2=0x7f58cc191bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.646+0000 7f58d264a700 1 --2- 172.21.15.7:0/564053559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc191bd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f58bc003f70 tx=0x7f58bc011b40 comp rx=0 tx=0).stop 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.646+0000 7f58d264a700 1 -- 172.21.15.7:0/564053559 shutdown_connections 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.646+0000 7f58d264a700 1 --2- 172.21.15.7:0/564053559 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f58b4038e90 0x7f58b403b350 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.646+0000 7f58d264a700 1 --2- 172.21.15.7:0/564053559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58cc106a60 0x7f58cc191bd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.646+0000 7f58d264a700 1 -- 172.21.15.7:0/564053559 >> 172.21.15.7:0/564053559 conn(0x7f58cc102040 msgr2=0x7f58cc102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.646+0000 7f58d264a700 1 -- 172.21.15.7:0/564053559 shutdown_connections 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:05.646+0000 7f58d264a700 1 -- 172.21.15.7:0/564053559 wait complete. 2024-03-23T13:10:06.193 INFO:teuthology.orchestra.run.smithi007.stdout:mgr is available 2024-03-23T13:10:06.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:05 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/564053559' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T13:10:07.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:06 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/27495673' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-23T13:10:07.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:10:07.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-23T13:10:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.577+0000 7ff9bd113700 1 Processor -- start 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.577+0000 7ff9bd113700 1 -- start start 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.577+0000 7ff9bd113700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b8106e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.577+0000 7ff9bd113700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b8107450 con 0x7ff9b8106a60 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.577+0000 7ff9b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b8106e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.577+0000 7ff9b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b8106e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46492/0 (socket says 172.21.15.7:46492) 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.577+0000 7ff9b6d9d700 1 -- 172.21.15.7:0/493794233 learned_addr learned my addr 172.21.15.7:0/493794233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.578+0000 7ff9b6d9d700 1 -- 172.21.15.7:0/493794233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b8107c60 con 0x7ff9b8106a60 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.578+0000 7ff9b6d9d700 1 --2- 172.21.15.7:0/493794233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b8106e80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80098a0 tx=0x7ff9a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3e7d0204097d62 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.578+0000 7ff9b5d9b700 1 -- 172.21.15.7:0/493794233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9a8004160 con 0x7ff9b8106a60 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.578+0000 7ff9b5d9b700 1 -- 172.21.15.7:0/493794233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9a80042c0 con 0x7ff9b8106a60 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.578+0000 7ff9b5d9b700 1 -- 172.21.15.7:0/493794233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9a8011ac0 con 0x7ff9b8106a60 2024-03-23T13:10:07.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.579+0000 7ff9bd113700 1 -- 172.21.15.7:0/493794233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 msgr2=0x7ff9b8106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.579+0000 7ff9bd113700 1 --2- 172.21.15.7:0/493794233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b8106e80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80098a0 tx=0x7ff9a8009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.579+0000 7ff9bd113700 1 -- 172.21.15.7:0/493794233 shutdown_connections 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.579+0000 7ff9bd113700 1 --2- 172.21.15.7:0/493794233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b8106e80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.579+0000 7ff9bd113700 1 -- 172.21.15.7:0/493794233 >> 172.21.15.7:0/493794233 conn(0x7ff9b8102040 msgr2=0x7ff9b8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.579+0000 7ff9bd113700 1 -- 172.21.15.7:0/493794233 shutdown_connections 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.579+0000 7ff9bd113700 1 -- 172.21.15.7:0/493794233 wait complete. 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.579+0000 7ff9bd113700 1 Processor -- start 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9bd113700 1 -- start start 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9bd113700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b807ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9bd113700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b807bf80 con 0x7ff9b8106a60 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b807ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b807ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46506/0 (socket says 172.21.15.7:46506) 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9b6d9d700 1 -- 172.21.15.7:0/27495673 learned_addr learned my addr 172.21.15.7:0/27495673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9b6d9d700 1 -- 172.21.15.7:0/27495673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a8009550 con 0x7ff9b8106a60 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9b6d9d700 1 --2- 172.21.15.7:0/27495673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b807ba40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8000c00 tx=0x7ff9a8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9affff700 1 -- 172.21.15.7:0/27495673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9a8003aa0 con 0x7ff9b8106a60 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9affff700 1 -- 172.21.15.7:0/27495673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9a8003c00 con 0x7ff9b8106a60 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.580+0000 7ff9affff700 1 -- 172.21.15.7:0/27495673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9a80248a0 con 0x7ff9b8106a60 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.581+0000 7ff9bd113700 1 -- 172.21.15.7:0/27495673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b819e6b0 con 0x7ff9b8106a60 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.581+0000 7ff9bd113700 1 -- 172.21.15.7:0/27495673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b819eb50 con 0x7ff9b8106a60 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.581+0000 7ff9affff700 1 -- 172.21.15.7:0/27495673 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 45917+0+0 (secure 0 0 0) 0x7ff9a802d3f0 con 0x7ff9b8106a60 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.582+0000 7ff9affff700 1 --2- 172.21.15.7:0/27495673 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7ff9a4038f30 0x7ff9a403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.582+0000 7ff9affff700 1 -- 172.21.15.7:0/27495673 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff9a804cd70 con 0x7ff9b8106a60 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.582+0000 7ff9b659c700 1 --2- 172.21.15.7:0/27495673 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7ff9a4038f30 0x7ff9a403b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.582+0000 7ff9bd113700 1 -- 172.21.15.7:0/27495673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b810ae90 con 0x7ff9b8106a60 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.582+0000 7ff9b659c700 1 --2- 172.21.15.7:0/27495673 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7ff9a4038f30 0x7ff9a403b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0006fd0 tx=0x7ff9a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.587+0000 7ff9affff700 1 -- 172.21.15.7:0/27495673 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff9a802d6a0 con 0x7ff9b8106a60 2024-03-23T13:10:07.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.789+0000 7ff9bd113700 1 -- 172.21.15.7:0/27495673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff9b810b020 con 0x7ff9b8106a60 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.789+0000 7ff9affff700 1 -- 172.21.15.7:0/27495673 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7ff9a800b3e0 con 0x7ff9b8106a60 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.792+0000 7ff9bd113700 1 -- 172.21.15.7:0/27495673 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7ff9a4038f30 msgr2=0x7ff9a403b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.792+0000 7ff9bd113700 1 --2- 172.21.15.7:0/27495673 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7ff9a4038f30 0x7ff9a403b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0006fd0 tx=0x7ff9a0006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.792+0000 7ff9bd113700 1 -- 172.21.15.7:0/27495673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 msgr2=0x7ff9b807ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.792+0000 7ff9bd113700 1 --2- 172.21.15.7:0/27495673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b807ba40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8000c00 tx=0x7ff9a8011b40 comp rx=0 tx=0).stop 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.792+0000 7ff9bd113700 1 -- 172.21.15.7:0/27495673 shutdown_connections 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.792+0000 7ff9bd113700 1 --2- 172.21.15.7:0/27495673 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7ff9a4038f30 0x7ff9a403b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.792+0000 7ff9bd113700 1 --2- 172.21.15.7:0/27495673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8106a60 0x7ff9b807ba40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.792+0000 7ff9bd113700 1 -- 172.21.15.7:0/27495673 >> 172.21.15.7:0/27495673 conn(0x7ff9b8102040 msgr2=0x7ff9b8102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.792+0000 7ff9bd113700 1 -- 172.21.15.7:0/27495673 shutdown_connections 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:06.792+0000 7ff9bd113700 1 -- 172.21.15.7:0/27495673 wait complete. 2024-03-23T13:10:07.359 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling cephadm module... 2024-03-23T13:10:08.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:08 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1584286535' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-23T13:10:08.317 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:08 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: ignoring --setuser ceph since I am not root 2024-03-23T13:10:08.317 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:08 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: ignoring --setgroup ceph since I am not root 2024-03-23T13:10:08.659 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:08 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:08.626+0000 7fc316825200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T13:10:08.667 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.732+0000 7f59bcb82700 1 Processor -- start 2024-03-23T13:10:08.667 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.732+0000 7f59bcb82700 1 -- start start 2024-03-23T13:10:08.667 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.732+0000 7f59bcb82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b8106e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:08.667 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.732+0000 7f59bcb82700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8107450 con 0x7f59b8106a60 2024-03-23T13:10:08.667 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.732+0000 7f59b659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b8106e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:08.667 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.732+0000 7f59b659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b8106e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46520/0 (socket says 172.21.15.7:46520) 2024-03-23T13:10:08.667 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.732+0000 7f59b659c700 1 -- 172.21.15.7:0/367018834 learned_addr learned my addr 172.21.15.7:0/367018834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:08.667 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.733+0000 7f59b659c700 1 -- 172.21.15.7:0/367018834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b8107c60 con 0x7f59b8106a60 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.733+0000 7f59b659c700 1 --2- 172.21.15.7:0/367018834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b8106e80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f59b00098a0 tx=0x7f59b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699c1115e3dce6ef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.733+0000 7f59b559a700 1 -- 172.21.15.7:0/367018834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b0004160 con 0x7f59b8106a60 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.733+0000 7f59b559a700 1 -- 172.21.15.7:0/367018834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59b00042c0 con 0x7f59b8106a60 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.733+0000 7f59b559a700 1 -- 172.21.15.7:0/367018834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b0011ac0 con 0x7f59b8106a60 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.734+0000 7f59bcb82700 1 -- 172.21.15.7:0/367018834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 msgr2=0x7f59b8106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.734+0000 7f59bcb82700 1 --2- 172.21.15.7:0/367018834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b8106e80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f59b00098a0 tx=0x7f59b0009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.734+0000 7f59bcb82700 1 -- 172.21.15.7:0/367018834 shutdown_connections 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.734+0000 7f59bcb82700 1 --2- 172.21.15.7:0/367018834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b8106e80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.734+0000 7f59bcb82700 1 -- 172.21.15.7:0/367018834 >> 172.21.15.7:0/367018834 conn(0x7f59b8102040 msgr2=0x7f59b8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.734+0000 7f59bcb82700 1 -- 172.21.15.7:0/367018834 shutdown_connections 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.734+0000 7f59bcb82700 1 -- 172.21.15.7:0/367018834 wait complete. 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.735+0000 7f59bcb82700 1 Processor -- start 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.735+0000 7f59bcb82700 1 -- start start 2024-03-23T13:10:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.735+0000 7f59bcb82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.735+0000 7f59bcb82700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59b819e090 con 0x7f59b8106a60 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.735+0000 7f59b659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.735+0000 7f59b659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46524/0 (socket says 172.21.15.7:46524) 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.735+0000 7f59b659c700 1 -- 172.21.15.7:0/1584286535 learned_addr learned my addr 172.21.15.7:0/1584286535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.735+0000 7f59b659c700 1 -- 172.21.15.7:0/1584286535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b0009550 con 0x7f59b8106a60 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.735+0000 7f59b659c700 1 --2- 172.21.15.7:0/1584286535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b819db50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f59b0003f70 tx=0x7f59b0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.735+0000 7f59af7fe700 1 -- 172.21.15.7:0/1584286535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b00039f0 con 0x7f59b8106a60 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.736+0000 7f59af7fe700 1 -- 172.21.15.7:0/1584286535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59b0003b50 con 0x7f59b8106a60 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.736+0000 7f59af7fe700 1 -- 172.21.15.7:0/1584286535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b001a690 con 0x7f59b8106a60 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.736+0000 7f59bcb82700 1 -- 172.21.15.7:0/1584286535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b819e290 con 0x7f59b8106a60 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.736+0000 7f59bcb82700 1 -- 172.21.15.7:0/1584286535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b819e730 con 0x7f59b8106a60 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.736+0000 7f59af7fe700 1 -- 172.21.15.7:0/1584286535 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 45917+0+0 (secure 0 0 0) 0x7f59b001a950 con 0x7f59b8106a60 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.736+0000 7f59af7fe700 1 --2- 172.21.15.7:0/1584286535 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f59a4038e40 0x7f59a403b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.737+0000 7f59af7fe700 1 -- 172.21.15.7:0/1584286535 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f59b004d2a0 con 0x7f59b8106a60 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.737+0000 7f59bcb82700 1 -- 172.21.15.7:0/1584286535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b81932f0 con 0x7f59b8106a60 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.737+0000 7f59b5d9b700 1 --2- 172.21.15.7:0/1584286535 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f59a4038e40 0x7f59a403b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.737+0000 7f59b5d9b700 1 --2- 172.21.15.7:0/1584286535 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f59a4038e40 0x7f59a403b300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59a0006fd0 tx=0x7f59a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:07.742+0000 7f59af7fe700 1 -- 172.21.15.7:0/1584286535 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f59b001ac00 con 0x7f59b8106a60 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.001+0000 7f59bcb82700 1 -- 172.21.15.7:0/1584286535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f59b802dcb0 con 0x7f59b8106a60 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.052+0000 7f59af7fe700 1 -- 172.21.15.7:0/1584286535 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f59b0018be0 con 0x7f59b8106a60 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.054+0000 7f59af7fe700 1 -- 172.21.15.7:0/1584286535 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 5) ==== 45928+0+0 (secure 0 0 0) 0x7f59b002d9b0 con 0x7f59b8106a60 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.055+0000 7f59bcb82700 1 -- 172.21.15.7:0/1584286535 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f59a4038e40 msgr2=0x7f59a403b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.055+0000 7f59bcb82700 1 --2- 172.21.15.7:0/1584286535 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f59a4038e40 0x7f59a403b300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59a0006fd0 tx=0x7f59a0006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.055+0000 7f59bcb82700 1 -- 172.21.15.7:0/1584286535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 msgr2=0x7f59b819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.055+0000 7f59bcb82700 1 --2- 172.21.15.7:0/1584286535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b819db50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f59b0003f70 tx=0x7f59b0011b40 comp rx=0 tx=0).stop 2024-03-23T13:10:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.055+0000 7f59bcb82700 1 -- 172.21.15.7:0/1584286535 shutdown_connections 2024-03-23T13:10:08.671 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.055+0000 7f59bcb82700 1 --2- 172.21.15.7:0/1584286535 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f59a4038e40 0x7f59a403b300 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:08.671 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.055+0000 7f59bcb82700 1 --2- 172.21.15.7:0/1584286535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8106a60 0x7f59b819db50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:08.671 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.055+0000 7f59bcb82700 1 -- 172.21.15.7:0/1584286535 >> 172.21.15.7:0/1584286535 conn(0x7f59b8102040 msgr2=0x7f59b8190430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:08.671 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.055+0000 7f59bcb82700 1 -- 172.21.15.7:0/1584286535 shutdown_connections 2024-03-23T13:10:08.671 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:08.055+0000 7f59bcb82700 1 -- 172.21.15.7:0/1584286535 wait complete. 2024-03-23T13:10:08.920 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:08 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:08.725+0000 7fc316825200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T13:10:09.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:09 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1584286535' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-23T13:10:09.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:09 smithi007 ceph-mon[32184]: mgrmap e5: x(active, since 6s) 2024-03-23T13:10:09.625 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:09.391+0000 7fc316825200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T13:10:09.625 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:09.491+0000 7fc316825200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T13:10:09.625 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:09.635+0000 7fc316825200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T13:10:09.999 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:09.733+0000 7fc316825200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T13:10:09.999 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:09.819+0000 7fc316825200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T13:10:10.007 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-23T13:10:10.007 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.122+0000 7f311b086700 1 Processor -- start 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.122+0000 7f311b086700 1 -- start start 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.123+0000 7f311b086700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f3114106250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.123+0000 7f311b086700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3114106820 con 0x7f3114105e30 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.123+0000 7f3118e22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f3114106250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.123+0000 7f3118e22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f3114106250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46544/0 (socket says 172.21.15.7:46544) 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.123+0000 7f3118e22700 1 -- 172.21.15.7:0/1334267086 learned_addr learned my addr 172.21.15.7:0/1334267086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.123+0000 7f3118e22700 1 -- 172.21.15.7:0/1334267086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3114106960 con 0x7f3114105e30 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.123+0000 7f3118e22700 1 --2- 172.21.15.7:0/1334267086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f3114106250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f31080098a0 tx=0x7f3108009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c63a25c90e49fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:10.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.123+0000 7f31137fe700 1 -- 172.21.15.7:0/1334267086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3108004160 con 0x7f3114105e30 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.123+0000 7f31137fe700 1 -- 172.21.15.7:0/1334267086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f31080042c0 con 0x7f3114105e30 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.123+0000 7f31137fe700 1 -- 172.21.15.7:0/1334267086 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3108011ac0 con 0x7f3114105e30 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.124+0000 7f311b086700 1 -- 172.21.15.7:0/1334267086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 msgr2=0x7f3114106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.124+0000 7f311b086700 1 --2- 172.21.15.7:0/1334267086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f3114106250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f31080098a0 tx=0x7f3108009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.124+0000 7f311b086700 1 -- 172.21.15.7:0/1334267086 shutdown_connections 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.124+0000 7f311b086700 1 --2- 172.21.15.7:0/1334267086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f3114106250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.124+0000 7f311b086700 1 -- 172.21.15.7:0/1334267086 >> 172.21.15.7:0/1334267086 conn(0x7f31141017a0 msgr2=0x7f3114103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.124+0000 7f311b086700 1 -- 172.21.15.7:0/1334267086 shutdown_connections 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.124+0000 7f311b086700 1 -- 172.21.15.7:0/1334267086 wait complete. 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.125+0000 7f311b086700 1 Processor -- start 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.125+0000 7f311b086700 1 -- start start 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.125+0000 7f311b086700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f311419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.125+0000 7f311b086700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f311419bed0 con 0x7f3114105e30 2024-03-23T13:10:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.125+0000 7f3118e22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f311419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.125+0000 7f3118e22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f311419b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46548/0 (socket says 172.21.15.7:46548) 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.125+0000 7f3118e22700 1 -- 172.21.15.7:0/3984022166 learned_addr learned my addr 172.21.15.7:0/3984022166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.125+0000 7f3118e22700 1 -- 172.21.15.7:0/3984022166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3108009550 con 0x7f3114105e30 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.125+0000 7f3118e22700 1 --2- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f311419b990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3108003f70 tx=0x7f3108011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.126+0000 7f3111ffb700 1 -- 172.21.15.7:0/3984022166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31080039f0 con 0x7f3114105e30 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.126+0000 7f3111ffb700 1 -- 172.21.15.7:0/3984022166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3108003b50 con 0x7f3114105e30 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.126+0000 7f3111ffb700 1 -- 172.21.15.7:0/3984022166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f310801a690 con 0x7f3114105e30 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.126+0000 7f311b086700 1 -- 172.21.15.7:0/3984022166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311419c0d0 con 0x7f3114105e30 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.126+0000 7f311b086700 1 -- 172.21.15.7:0/3984022166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311419c570 con 0x7f3114105e30 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.127+0000 7f3111ffb700 1 -- 172.21.15.7:0/3984022166 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 45928+0+0 (secure 0 0 0) 0x7f310801a900 con 0x7f3114105e30 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.127+0000 7f3111ffb700 1 --2- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f3104038ee0 0x7f310403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.127+0000 7f3113fff700 1 -- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f3104038ee0 msgr2=0x7f310403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/310337452 2024-03-23T13:10:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.127+0000 7f3111ffb700 1 -- 172.21.15.7:0/3984022166 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f310804d0f0 con 0x7f3114105e30 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.127+0000 7f3113fff700 1 --2- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f3104038ee0 0x7f310403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.127+0000 7f311b086700 1 -- 172.21.15.7:0/3984022166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3114063830 con 0x7f3114105e30 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.132+0000 7f3111ffb700 1 -- 172.21.15.7:0/3984022166 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f310801abb0 con 0x7f3114105e30 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.328+0000 7f3113fff700 1 -- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f3104038ee0 msgr2=0x7f310403b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/310337452 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.328+0000 7f3113fff700 1 --2- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f3104038ee0 0x7f310403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.396+0000 7f311b086700 1 -- 172.21.15.7:0/3984022166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f311419f0b0 con 0x7f3114105e30 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.396+0000 7f3111ffb700 1 -- 172.21.15.7:0/3984022166 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f310804a030 con 0x7f3114105e30 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.398+0000 7f311b086700 1 -- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f3104038ee0 msgr2=0x7f310403b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.398+0000 7f311b086700 1 --2- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f3104038ee0 0x7f310403b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.398+0000 7f311b086700 1 -- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 msgr2=0x7f311419b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.398+0000 7f311b086700 1 --2- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f311419b990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3108003f70 tx=0x7f3108011b40 comp rx=0 tx=0).stop 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.398+0000 7f311b086700 1 -- 172.21.15.7:0/3984022166 shutdown_connections 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.398+0000 7f311b086700 1 --2- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f3104038ee0 0x7f310403b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:10.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.398+0000 7f311b086700 1 --2- 172.21.15.7:0/3984022166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3114105e30 0x7f311419b990 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:10.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.398+0000 7f311b086700 1 -- 172.21.15.7:0/3984022166 >> 172.21.15.7:0/3984022166 conn(0x7f31141017a0 msgr2=0x7f31141022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:10.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.399+0000 7f311b086700 1 -- 172.21.15.7:0/3984022166 shutdown_connections 2024-03-23T13:10:10.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:09.399+0000 7f311b086700 1 -- 172.21.15.7:0/3984022166 wait complete. 2024-03-23T13:10:10.012 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-03-23T13:10:10.012 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 5... 2024-03-23T13:10:10.281 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:10 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3984022166' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-23T13:10:10.281 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:10.091+0000 7fc316825200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T13:10:10.281 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:10.205+0000 7fc316825200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T13:10:10.567 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:10.298+0000 7fc316825200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T13:10:10.567 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:10.396+0000 7fc316825200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T13:10:11.067 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:10.578+0000 7fc316825200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T13:10:11.817 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:11 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:11.567+0000 7fc316825200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T13:10:11.817 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:11 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:11.701+0000 7fc316825200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T13:10:12.317 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:12.039+0000 7fc316825200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T13:10:12.317 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:12.131+0000 7fc316825200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T13:10:12.806 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:12.332+0000 7fc316825200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T13:10:12.806 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:12.524+0000 7fc316825200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T13:10:13.066 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:12.815+0000 7fc316825200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T13:10:13.066 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:13 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:13.076+0000 7fc316825200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T13:10:13.317 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:13 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:13.319+0000 7fc316825200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T13:10:14.567 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:14.148+0000 7fc316825200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T13:10:15.039 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:14.726+0000 7fc316825200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T13:10:15.039 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:14.826+0000 7fc316825200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T13:10:15.039 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:15.049+0000 7fc316825200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T13:10:15.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: Active manager daemon x restarted 2024-03-23T13:10:15.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: Activating manager daemon x 2024-03-23T13:10:15.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: osdmap e2: 0 total, 0 up, 0 in 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: mgrmap e6: x(active, starting, since 0.00355217s) 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: Manager daemon x is now available 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: Found migration_current of "None". Setting to last migration. 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:15.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:15 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:16.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:16 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:16.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:16 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T13:10:16.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:16 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T13:10:16.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:16 smithi007 ceph-mon[32184]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-23T13:10:16.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:16 smithi007 ceph-mon[32184]: mgrmap e7: x(active, since 1.00534s) 2024-03-23T13:10:16.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:16 smithi007 ceph-mon[32184]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-23T13:10:16.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.516+0000 7f39e6b06700 1 Processor -- start 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.516+0000 7f39e6b06700 1 -- start start 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.516+0000 7f39e6b06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e01084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.516+0000 7f39e6b06700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0108ab0 con 0x7f39e01080c0 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.516+0000 7f39e48a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e01084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.516+0000 7f39e48a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e01084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46558/0 (socket says 172.21.15.7:46558) 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.516+0000 7f39e48a2700 1 -- 172.21.15.7:0/4187269740 learned_addr learned my addr 172.21.15.7:0/4187269740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.517+0000 7f39e48a2700 1 -- 172.21.15.7:0/4187269740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e0108bf0 con 0x7f39e01080c0 2024-03-23T13:10:16.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.517+0000 7f39e48a2700 1 --2- 172.21.15.7:0/4187269740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e01084e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f39d40098a0 tx=0x7f39d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f79ef659ee7d27 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.517+0000 7f39df7fe700 1 -- 172.21.15.7:0/4187269740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39d4004160 con 0x7f39e01080c0 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.517+0000 7f39df7fe700 1 -- 172.21.15.7:0/4187269740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f39d40042c0 con 0x7f39e01080c0 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.517+0000 7f39df7fe700 1 -- 172.21.15.7:0/4187269740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39d4011ac0 con 0x7f39e01080c0 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.518+0000 7f39e6b06700 1 -- 172.21.15.7:0/4187269740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 msgr2=0x7f39e01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.518+0000 7f39e6b06700 1 --2- 172.21.15.7:0/4187269740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e01084e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f39d40098a0 tx=0x7f39d4009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.518+0000 7f39e6b06700 1 -- 172.21.15.7:0/4187269740 shutdown_connections 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.518+0000 7f39e6b06700 1 --2- 172.21.15.7:0/4187269740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e01084e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.518+0000 7f39e6b06700 1 -- 172.21.15.7:0/4187269740 >> 172.21.15.7:0/4187269740 conn(0x7f39e01017a0 msgr2=0x7f39e0103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.518+0000 7f39e6b06700 1 -- 172.21.15.7:0/4187269740 shutdown_connections 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.518+0000 7f39e6b06700 1 -- 172.21.15.7:0/4187269740 wait complete. 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.519+0000 7f39e6b06700 1 Processor -- start 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.519+0000 7f39e6b06700 1 -- start start 2024-03-23T13:10:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.519+0000 7f39e6b06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.519+0000 7f39e6b06700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39e019e040 con 0x7f39e01080c0 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.519+0000 7f39e48a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.519+0000 7f39e48a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e019db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46564/0 (socket says 172.21.15.7:46564) 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.519+0000 7f39e48a2700 1 -- 172.21.15.7:0/1500134296 learned_addr learned my addr 172.21.15.7:0/1500134296 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.519+0000 7f39e48a2700 1 -- 172.21.15.7:0/1500134296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d4009550 con 0x7f39e01080c0 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.519+0000 7f39e48a2700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e019db00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009130 tx=0x7f39d4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.520+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39d4003aa0 con 0x7f39e01080c0 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.520+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f39d4003c00 con 0x7f39e01080c0 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.520+0000 7f39e6b06700 1 -- 172.21.15.7:0/1500134296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e019e240 con 0x7f39e01080c0 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.520+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39d4024800 con 0x7f39e01080c0 2024-03-23T13:10:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.520+0000 7f39e6b06700 1 -- 172.21.15.7:0/1500134296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e019e6e0 con 0x7f39e01080c0 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.521+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 45928+0+0 (secure 0 0 0) 0x7f39d401abd0 con 0x7f39e01080c0 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.521+0000 7f39ddffb700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 0x7f39d003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.521+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 --> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f39d003bab0 con 0x7f39d0038ee0 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.521+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f39d404ce00 con 0x7f39e01080c0 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.521+0000 7f39dffff700 1 -- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 msgr2=0x7f39d003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/310337452 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.521+0000 7f39dffff700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 0x7f39d003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.722+0000 7f39dffff700 1 -- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 msgr2=0x7f39d003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/310337452 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:10.722+0000 7f39dffff700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 0x7f39d003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:11.123+0000 7f39dffff700 1 -- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 msgr2=0x7f39d003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/310337452 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:11.123+0000 7f39dffff700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 0x7f39d003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:11.924+0000 7f39dffff700 1 -- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 msgr2=0x7f39d003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/310337452 2024-03-23T13:10:16.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:11.924+0000 7f39dffff700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 0x7f39d003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:13.525+0000 7f39dffff700 1 -- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 msgr2=0x7f39d003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/310337452 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:13.525+0000 7f39dffff700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 0x7f39d003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:15.064+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 6) ==== 45695+0+0 (secure 0 0 0) 0x7f39d4024ca0 con 0x7f39e01080c0 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:15.064+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 msgr2=0x7f39d003b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:15.064+0000 7f39ddffb700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 0x7f39d003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.065+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 7) ==== 45822+0+0 (secure 0 0 0) 0x7f39d404e5f0 con 0x7f39e01080c0 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.065+0000 7f39ddffb700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f39d003c7d0 0x7f39d003ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.065+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 --> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f39d003bab0 con 0x7f39d003c7d0 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.065+0000 7f39dffff700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f39d003c7d0 0x7f39d003ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.066+0000 7f39dffff700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f39d003c7d0 0x7f39d003ebc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f39cc003a10 tx=0x7f39cc009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:16.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.067+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 <== mgr.14120 v2:172.21.15.7:6800/3945637763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f39d003bab0 con 0x7f39d003c7d0 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.078+0000 7f39e6b06700 1 -- 172.21.15.7:0/1500134296 --> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f39e0106890 con 0x7f39d003c7d0 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.078+0000 7f39ddffb700 1 -- 172.21.15.7:0/1500134296 <== mgr.14120 v2:172.21.15.7:6800/3945637763 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f39e0106890 con 0x7f39d003c7d0 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.078+0000 7f39e6b06700 1 -- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f39d003c7d0 msgr2=0x7f39d003ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.078+0000 7f39e6b06700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f39d003c7d0 0x7f39d003ebc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f39cc003a10 tx=0x7f39cc009290 comp rx=0 tx=0).stop 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.078+0000 7f39e6b06700 1 -- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 msgr2=0x7f39e019db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.078+0000 7f39e6b06700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e019db00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009130 tx=0x7f39d4011b40 comp rx=0 tx=0).stop 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.079+0000 7f39e6b06700 1 -- 172.21.15.7:0/1500134296 shutdown_connections 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.079+0000 7f39e6b06700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f39d003c7d0 0x7f39d003ebc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.079+0000 7f39e6b06700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e01080c0 0x7f39e019db00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.079+0000 7f39e6b06700 1 --2- 172.21.15.7:0/1500134296 >> [v2:172.21.15.7:6800/310337452,v1:172.21.15.7:6801/310337452] conn(0x7f39d0038ee0 0x7f39d003b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.079+0000 7f39e6b06700 1 -- 172.21.15.7:0/1500134296 >> 172.21.15.7:0/1500134296 conn(0x7f39e01017a0 msgr2=0x7f39e0194330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:16.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.079+0000 7f39e6b06700 1 -- 172.21.15.7:0/1500134296 shutdown_connections 2024-03-23T13:10:16.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:16.079+0000 7f39e6b06700 1 -- 172.21.15.7:0/1500134296 wait complete. 2024-03-23T13:10:16.681 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 5 is available 2024-03-23T13:10:16.681 INFO:teuthology.orchestra.run.smithi007.stdout:Setting orchestrator backend to cephadm... 2024-03-23T13:10:17.567 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:17 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:17.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:17 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:17.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:17 smithi007 ceph-mon[32184]: [23/Mar/2024:13:10:16] ENGINE Bus STARTING 2024-03-23T13:10:17.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:17 smithi007 ceph-mon[32184]: [23/Mar/2024:13:10:16] ENGINE Serving on http://172.21.15.7:8765 2024-03-23T13:10:17.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:17 smithi007 ceph-mon[32184]: [23/Mar/2024:13:10:17] ENGINE Serving on https://172.21.15.7:7150 2024-03-23T13:10:17.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:17 smithi007 ceph-mon[32184]: [23/Mar/2024:13:10:17] ENGINE Bus STARTED 2024-03-23T13:10:17.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:17 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:17.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:17 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:17.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:17 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:17.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.094+0000 7faeffb67700 1 Processor -- start 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.094+0000 7faeffb67700 1 -- start start 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.094+0000 7faeffb67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef8106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.094+0000 7faeffb67700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faef807be40 con 0x7faef8106420 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.094+0000 7faefd903700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef8106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.094+0000 7faefd903700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef8106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51040/0 (socket says 172.21.15.7:51040) 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.094+0000 7faefd903700 1 -- 172.21.15.7:0/109266535 learned_addr learned my addr 172.21.15.7:0/109266535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.095+0000 7faefd903700 1 -- 172.21.15.7:0/109266535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef8106d80 con 0x7faef8106420 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.095+0000 7faefd903700 1 --2- 172.21.15.7:0/109266535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef8106840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faef0009e80 tx=0x7faef000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3eebce08dac2fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.095+0000 7faefc901700 1 -- 172.21.15.7:0/109266535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef000e070 con 0x7faef8106420 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.095+0000 7faefc901700 1 -- 172.21.15.7:0/109266535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faef000b8c0 con 0x7faef8106420 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.095+0000 7faefc901700 1 -- 172.21.15.7:0/109266535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef0004550 con 0x7faef8106420 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.095+0000 7faeffb67700 1 -- 172.21.15.7:0/109266535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 msgr2=0x7faef8106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.095+0000 7faeffb67700 1 --2- 172.21.15.7:0/109266535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef8106840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faef0009e80 tx=0x7faef000b330 comp rx=0 tx=0).stop 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.096+0000 7faeffb67700 1 -- 172.21.15.7:0/109266535 shutdown_connections 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.096+0000 7faeffb67700 1 --2- 172.21.15.7:0/109266535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef8106840 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.096+0000 7faeffb67700 1 -- 172.21.15.7:0/109266535 >> 172.21.15.7:0/109266535 conn(0x7faef8102040 msgr2=0x7faef8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.096+0000 7faeffb67700 1 -- 172.21.15.7:0/109266535 shutdown_connections 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.096+0000 7faeffb67700 1 -- 172.21.15.7:0/109266535 wait complete. 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.096+0000 7faeffb67700 1 Processor -- start 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.096+0000 7faeffb67700 1 -- start start 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.096+0000 7faeffb67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.096+0000 7faeffb67700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faef819dfb0 con 0x7faef8106420 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.097+0000 7faefd903700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.097+0000 7faefd903700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef819da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51044/0 (socket says 172.21.15.7:51044) 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.097+0000 7faefd903700 1 -- 172.21.15.7:0/106794458 learned_addr learned my addr 172.21.15.7:0/106794458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.097+0000 7faefd903700 1 -- 172.21.15.7:0/106794458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef0009b30 con 0x7faef8106420 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.097+0000 7faefd903700 1 --2- 172.21.15.7:0/106794458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef819da70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faef0009fd0 tx=0x7faef0003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.097+0000 7faef6ffd700 1 -- 172.21.15.7:0/106794458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef000e040 con 0x7faef8106420 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.097+0000 7faef6ffd700 1 -- 172.21.15.7:0/106794458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faef00128f0 con 0x7faef8106420 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.097+0000 7faeffb67700 1 -- 172.21.15.7:0/106794458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef819e1b0 con 0x7faef8106420 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.097+0000 7faef6ffd700 1 -- 172.21.15.7:0/106794458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef001c890 con 0x7faef8106420 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.097+0000 7faeffb67700 1 -- 172.21.15.7:0/106794458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef819e630 con 0x7faef8106420 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.098+0000 7faef6ffd700 1 -- 172.21.15.7:0/106794458 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 7) ==== 45822+0+0 (secure 0 0 0) 0x7faef001c9f0 con 0x7faef8106420 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.098+0000 7faeffb67700 1 -- 172.21.15.7:0/106794458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef8197700 con 0x7faef8106420 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.098+0000 7faef6ffd700 1 --2- 172.21.15.7:0/106794458 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7faee0038e10 0x7faee003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.098+0000 7faef6ffd700 1 -- 172.21.15.7:0/106794458 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faef004cf20 con 0x7faef8106420 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.098+0000 7faefd102700 1 --2- 172.21.15.7:0/106794458 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7faee0038e10 0x7faee003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.099+0000 7faefd102700 1 --2- 172.21.15.7:0/106794458 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7faee0038e10 0x7faee003b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faeec006fd0 tx=0x7faeec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.103+0000 7faef6ffd700 1 -- 172.21.15.7:0/106794458 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7faef001b720 con 0x7faef8106420 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.322+0000 7faeffb67700 1 -- 172.21.15.7:0/106794458 --> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7faef802dd70 con 0x7faee0038e10 2024-03-23T13:10:17.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.328+0000 7faef6ffd700 1 -- 172.21.15.7:0/106794458 <== mgr.14120 v2:172.21.15.7:6800/3945637763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faef802dd70 con 0x7faee0038e10 2024-03-23T13:10:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.330+0000 7faeffb67700 1 -- 172.21.15.7:0/106794458 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7faee0038e10 msgr2=0x7faee003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.330+0000 7faeffb67700 1 --2- 172.21.15.7:0/106794458 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7faee0038e10 0x7faee003b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faeec006fd0 tx=0x7faeec006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.330+0000 7faeffb67700 1 -- 172.21.15.7:0/106794458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 msgr2=0x7faef819da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.330+0000 7faeffb67700 1 --2- 172.21.15.7:0/106794458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef819da70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faef0009fd0 tx=0x7faef0003d70 comp rx=0 tx=0).stop 2024-03-23T13:10:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.330+0000 7faeffb67700 1 -- 172.21.15.7:0/106794458 shutdown_connections 2024-03-23T13:10:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.330+0000 7faeffb67700 1 --2- 172.21.15.7:0/106794458 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7faee0038e10 0x7faee003b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.330+0000 7faeffb67700 1 --2- 172.21.15.7:0/106794458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106420 0x7faef819da70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.330+0000 7faeffb67700 1 -- 172.21.15.7:0/106794458 >> 172.21.15.7:0/106794458 conn(0x7faef8102040 msgr2=0x7faef8194840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.331+0000 7faeffb67700 1 -- 172.21.15.7:0/106794458 shutdown_connections 2024-03-23T13:10:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:17.331+0000 7faeffb67700 1 -- 172.21.15.7:0/106794458 wait complete. 2024-03-23T13:10:18.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:18 smithi007 ceph-mon[32184]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:10:18.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:18 smithi007 ceph-mon[32184]: mgrmap e8: x(active, since 2s) 2024-03-23T13:10:19.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.337+0000 7f8445b8d700 1 Processor -- start 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.337+0000 7f8445b8d700 1 -- start start 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.337+0000 7f8445b8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f8440106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.337+0000 7f8445b8d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f844007be40 con 0x7f8440106420 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.337+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f8440106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.337+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f8440106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51048/0 (socket says 172.21.15.7:51048) 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.337+0000 7f843f7fe700 1 -- 172.21.15.7:0/2796232541 learned_addr learned my addr 172.21.15.7:0/2796232541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.338+0000 7f843f7fe700 1 -- 172.21.15.7:0/2796232541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8440106d80 con 0x7f8440106420 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.338+0000 7f843f7fe700 1 --2- 172.21.15.7:0/2796232541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f8440106840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f84300098a0 tx=0x7f8430009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5744217f17130d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.338+0000 7f843e7fc700 1 -- 172.21.15.7:0/2796232541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8430004160 con 0x7f8440106420 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.338+0000 7f843e7fc700 1 -- 172.21.15.7:0/2796232541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f84300042c0 con 0x7f8440106420 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.338+0000 7f843e7fc700 1 -- 172.21.15.7:0/2796232541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8430011ac0 con 0x7f8440106420 2024-03-23T13:10:19.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.338+0000 7f8445b8d700 1 -- 172.21.15.7:0/2796232541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 msgr2=0x7f8440106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.338+0000 7f8445b8d700 1 --2- 172.21.15.7:0/2796232541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f8440106840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f84300098a0 tx=0x7f8430009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.338+0000 7f8445b8d700 1 -- 172.21.15.7:0/2796232541 shutdown_connections 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.338+0000 7f8445b8d700 1 --2- 172.21.15.7:0/2796232541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f8440106840 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.338+0000 7f8445b8d700 1 -- 172.21.15.7:0/2796232541 >> 172.21.15.7:0/2796232541 conn(0x7f8440102040 msgr2=0x7f8440104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.339+0000 7f8445b8d700 1 -- 172.21.15.7:0/2796232541 shutdown_connections 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.339+0000 7f8445b8d700 1 -- 172.21.15.7:0/2796232541 wait complete. 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.339+0000 7f8445b8d700 1 Processor -- start 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.339+0000 7f8445b8d700 1 -- start start 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f8445b8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f844019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f8445b8d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f844019e020 con 0x7f8440106420 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f844019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f844019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51062/0 (socket says 172.21.15.7:51062) 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f843f7fe700 1 -- 172.21.15.7:0/1638278592 learned_addr learned my addr 172.21.15.7:0/1638278592 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:19.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f843f7fe700 1 -- 172.21.15.7:0/1638278592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8430009550 con 0x7f8440106420 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f843f7fe700 1 --2- 172.21.15.7:0/1638278592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f844019dae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8430003f70 tx=0x7f8430011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f843cff9700 1 -- 172.21.15.7:0/1638278592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84300039f0 con 0x7f8440106420 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f843cff9700 1 -- 172.21.15.7:0/1638278592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8430003b50 con 0x7f8440106420 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f8445b8d700 1 -- 172.21.15.7:0/1638278592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844019e220 con 0x7f8440106420 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f843cff9700 1 -- 172.21.15.7:0/1638278592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f843001a760 con 0x7f8440106420 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.340+0000 7f8445b8d700 1 -- 172.21.15.7:0/1638278592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844019e6c0 con 0x7f8440106420 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.341+0000 7f843cff9700 1 -- 172.21.15.7:0/1638278592 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f843001a9f0 con 0x7f8440106420 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.341+0000 7f8445b8d700 1 -- 172.21.15.7:0/1638278592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84401977a0 con 0x7f8440106420 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.341+0000 7f843cff9700 1 --2- 172.21.15.7:0/1638278592 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f8428038e90 0x7f842803b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.341+0000 7f843cff9700 1 -- 172.21.15.7:0/1638278592 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f843004d3f0 con 0x7f8440106420 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.342+0000 7f843effd700 1 --2- 172.21.15.7:0/1638278592 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f8428038e90 0x7f842803b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.342+0000 7f843effd700 1 --2- 172.21.15.7:0/1638278592 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f8428038e90 0x7f842803b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8434006fd0 tx=0x7f8434006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.347+0000 7f843cff9700 1 -- 172.21.15.7:0/1638278592 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f843001aca0 con 0x7f8440106420 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.553+0000 7f8445b8d700 1 -- 172.21.15.7:0/1638278592 --> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f844002ddc0 con 0x7f8428038e90 2024-03-23T13:10:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.553+0000 7f843cff9700 1 -- 172.21.15.7:0/1638278592 <== mgr.14120 v2:172.21.15.7:6800/3945637763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f844002ddc0 con 0x7f8428038e90 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.555+0000 7f8445b8d700 1 -- 172.21.15.7:0/1638278592 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f8428038e90 msgr2=0x7f842803b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.555+0000 7f8445b8d700 1 --2- 172.21.15.7:0/1638278592 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f8428038e90 0x7f842803b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8434006fd0 tx=0x7f8434006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.555+0000 7f8445b8d700 1 -- 172.21.15.7:0/1638278592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 msgr2=0x7f844019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.555+0000 7f8445b8d700 1 --2- 172.21.15.7:0/1638278592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f844019dae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8430003f70 tx=0x7f8430011b40 comp rx=0 tx=0).stop 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.555+0000 7f8445b8d700 1 -- 172.21.15.7:0/1638278592 shutdown_connections 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.555+0000 7f8445b8d700 1 --2- 172.21.15.7:0/1638278592 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f8428038e90 0x7f842803b350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.555+0000 7f8445b8d700 1 --2- 172.21.15.7:0/1638278592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8440106420 0x7f844019dae0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.555+0000 7f8445b8d700 1 -- 172.21.15.7:0/1638278592 >> 172.21.15.7:0/1638278592 conn(0x7f8440102040 msgr2=0x7f84401948e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.555+0000 7f8445b8d700 1 -- 172.21.15.7:0/1638278592 shutdown_connections 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:18.555+0000 7f8445b8d700 1 -- 172.21.15.7:0/1638278592 wait complete. 2024-03-23T13:10:19.121 INFO:teuthology.orchestra.run.smithi007.stdout:Generating ssh key... 2024-03-23T13:10:19.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-mon[32184]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: Generating public/private rsa key pair. 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: Your identification has been saved in /tmp/tmpjrtev_t8/key. 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: Your public key has been saved in /tmp/tmpjrtev_t8/key.pub. 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: The key fingerprint is: 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: SHA256:ncylvJy2YiaF7/pVnXqSWm72nh+zJysmVetiD6VqXa4 ceph-745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: The key's randomart image is: 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: +---[RSA 3072]----+ 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: | | 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: | | 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: | . | 2024-03-23T13:10:20.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: | = + ... | 2024-03-23T13:10:20.069 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: | .S B ..+. | 2024-03-23T13:10:20.069 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: | . .. +.=o | 2024-03-23T13:10:20.069 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: | o *oB+.o | 2024-03-23T13:10:20.069 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: | . =oo*O++.=| 2024-03-23T13:10:20.069 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: | .Boo+*EB*=o| 2024-03-23T13:10:20.069 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: +----[SHA256]-----+ 2024-03-23T13:10:20.566 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.563+0000 7f264ee74700 1 Processor -- start 2024-03-23T13:10:20.566 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.564+0000 7f264ee74700 1 -- start start 2024-03-23T13:10:20.566 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.564+0000 7f264ee74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f2648108000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:20.566 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.564+0000 7f264ee74700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26481085d0 con 0x7f2648107be0 2024-03-23T13:10:20.566 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.564+0000 7f264cc10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f2648108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:20.566 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.564+0000 7f264cc10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f2648108000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51064/0 (socket says 172.21.15.7:51064) 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.564+0000 7f264cc10700 1 -- 172.21.15.7:0/1926837241 learned_addr learned my addr 172.21.15.7:0/1926837241 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.564+0000 7f264cc10700 1 -- 172.21.15.7:0/1926837241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2648108d40 con 0x7f2648107be0 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.564+0000 7f264cc10700 1 --2- 172.21.15.7:0/1926837241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f2648108000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2640009e80 tx=0x7f264000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03c72ce603ca3c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.565+0000 7f26477fe700 1 -- 172.21.15.7:0/1926837241 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f264000e070 con 0x7f2648107be0 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.565+0000 7f26477fe700 1 -- 172.21.15.7:0/1926837241 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f264000b8c0 con 0x7f2648107be0 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.565+0000 7f26477fe700 1 -- 172.21.15.7:0/1926837241 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2640004550 con 0x7f2648107be0 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.565+0000 7f264ee74700 1 -- 172.21.15.7:0/1926837241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 msgr2=0x7f2648108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.565+0000 7f264ee74700 1 --2- 172.21.15.7:0/1926837241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f2648108000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2640009e80 tx=0x7f264000b330 comp rx=0 tx=0).stop 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.565+0000 7f264ee74700 1 -- 172.21.15.7:0/1926837241 shutdown_connections 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.565+0000 7f264ee74700 1 --2- 172.21.15.7:0/1926837241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f2648108000 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.565+0000 7f264ee74700 1 -- 172.21.15.7:0/1926837241 >> 172.21.15.7:0/1926837241 conn(0x7f2648103200 msgr2=0x7f2648105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.565+0000 7f264ee74700 1 -- 172.21.15.7:0/1926837241 shutdown_connections 2024-03-23T13:10:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.565+0000 7f264ee74700 1 -- 172.21.15.7:0/1926837241 wait complete. 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.566+0000 7f264ee74700 1 Processor -- start 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.566+0000 7f264ee74700 1 -- start start 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.566+0000 7f264ee74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f264819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.566+0000 7f264ee74700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f264819bda0 con 0x7f2648107be0 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.566+0000 7f264cc10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f264819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.566+0000 7f264cc10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f264819b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51070/0 (socket says 172.21.15.7:51070) 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.566+0000 7f264cc10700 1 -- 172.21.15.7:0/2129305942 learned_addr learned my addr 172.21.15.7:0/2129305942 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.566+0000 7f264cc10700 1 -- 172.21.15.7:0/2129305942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2640009b30 con 0x7f2648107be0 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.567+0000 7f264cc10700 1 --2- 172.21.15.7:0/2129305942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f264819b860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2640009fd0 tx=0x7f2640003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.567+0000 7f2645ffb700 1 -- 172.21.15.7:0/2129305942 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f264000e040 con 0x7f2648107be0 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.567+0000 7f2645ffb700 1 -- 172.21.15.7:0/2129305942 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26400128f0 con 0x7f2648107be0 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.567+0000 7f264ee74700 1 -- 172.21.15.7:0/2129305942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264819bfa0 con 0x7f2648107be0 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.567+0000 7f2645ffb700 1 -- 172.21.15.7:0/2129305942 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f264001c890 con 0x7f2648107be0 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.567+0000 7f264ee74700 1 -- 172.21.15.7:0/2129305942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264819c420 con 0x7f2648107be0 2024-03-23T13:10:20.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.568+0000 7f2645ffb700 1 -- 172.21.15.7:0/2129305942 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f264001c9f0 con 0x7f2648107be0 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.568+0000 7f264ee74700 1 -- 172.21.15.7:0/2129305942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264810bf70 con 0x7f2648107be0 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.568+0000 7f2645ffb700 1 --2- 172.21.15.7:0/2129305942 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f2634038f30 0x7f263403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.568+0000 7f2645ffb700 1 -- 172.21.15.7:0/2129305942 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f264004cf60 con 0x7f2648107be0 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.568+0000 7f2647fff700 1 --2- 172.21.15.7:0/2129305942 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f2634038f30 0x7f263403b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.569+0000 7f2647fff700 1 --2- 172.21.15.7:0/2129305942 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f2634038f30 0x7f263403b3f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2638006fd0 tx=0x7f2638006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.573+0000 7f2645ffb700 1 -- 172.21.15.7:0/2129305942 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2640011940 con 0x7f2648107be0 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.777+0000 7f264ee74700 1 -- 172.21.15.7:0/2129305942 --> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f2648104360 con 0x7f2634038f30 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.956+0000 7f2645ffb700 1 -- 172.21.15.7:0/2129305942 <== mgr.14120 v2:172.21.15.7:6800/3945637763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2648104360 con 0x7f2634038f30 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.958+0000 7f264ee74700 1 -- 172.21.15.7:0/2129305942 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f2634038f30 msgr2=0x7f263403b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.958+0000 7f264ee74700 1 --2- 172.21.15.7:0/2129305942 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f2634038f30 0x7f263403b3f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2638006fd0 tx=0x7f2638006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.958+0000 7f264ee74700 1 -- 172.21.15.7:0/2129305942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 msgr2=0x7f264819b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.958+0000 7f264ee74700 1 --2- 172.21.15.7:0/2129305942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f264819b860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2640009fd0 tx=0x7f2640003d70 comp rx=0 tx=0).stop 2024-03-23T13:10:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.958+0000 7f264ee74700 1 -- 172.21.15.7:0/2129305942 shutdown_connections 2024-03-23T13:10:20.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.958+0000 7f264ee74700 1 --2- 172.21.15.7:0/2129305942 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f2634038f30 0x7f263403b3f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:20.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.958+0000 7f264ee74700 1 --2- 172.21.15.7:0/2129305942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648107be0 0x7f264819b860 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:20.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.958+0000 7f264ee74700 1 -- 172.21.15.7:0/2129305942 >> 172.21.15.7:0/2129305942 conn(0x7f2648103200 msgr2=0x7f2648103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:20.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.958+0000 7f264ee74700 1 -- 172.21.15.7:0/2129305942 shutdown_connections 2024-03-23T13:10:20.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:19.958+0000 7f264ee74700 1 -- 172.21.15.7:0/2129305942 wait complete. 2024-03-23T13:10:21.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:20 smithi007 ceph-mon[32184]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:10:21.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:20 smithi007 ceph-mon[32184]: Generating ssh key... 2024-03-23T13:10:21.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:20 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:21.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:20 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:21.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxg0h2LByN/B4Tw3qoUrmdrESxVNkRctLZadEsBY2ltlJP8fFMowGqm6UoWaOJoZd4CQYYYWpmlr58tQM1J/MJGSKHV1AknLIMz3K8YD3vR/4SEZoi7bEawMkxlQPGIxhT+e5jIhvHwmlEcch+F7LCMxuaB1Le5hcrQhAUIeYEv7oJqPM31FvYBvQJeDpePZodv2DAgdzyL0w8XoOORnEd2VGK1iJzgf0e2f+EWQoa0ihCmqvjSC+KBhGnt8wzdFQKkYI3YqUMrCLh4LJ3bZQDE5y1+o8bgih753PC2Zzckmnff8U1N3V8dxMjiMajOOqwyYsfaImcEqUqsVXGfEDfdyFe14VcwuHcfB2yJ1Sq6JHesM8Utz5ZDHOhajDj648gAEYLOxdh7EptRONGoT5Tet+JHuYkiEJi4OxAmg6A0lEu9pI5J/xfJXIPpYXqowpe7uOs+toWKMOFMO0haIKQD6OJY3gSs8ZDX800PL4XM170V/PwigVoUP9c+2tmoM8= ceph-745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:10:21.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.978+0000 7f1801a67700 1 Processor -- start 2024-03-23T13:10:21.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.978+0000 7f1801a67700 1 -- start start 2024-03-23T13:10:21.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.979+0000 7f1801a67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc10aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:21.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.979+0000 7f1801a67700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc07be20 con 0x7f17fc108650 2024-03-23T13:10:21.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.979+0000 7f17fb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc10aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:21.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.979+0000 7f17fb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc10aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51084/0 (socket says 172.21.15.7:51084) 2024-03-23T13:10:21.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.979+0000 7f17fb7fe700 1 -- 172.21.15.7:0/3350711849 learned_addr learned my addr 172.21.15.7:0/3350711849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:21.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.979+0000 7f17fb7fe700 1 -- 172.21.15.7:0/3350711849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc10af80 con 0x7f17fc108650 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.979+0000 7f17fb7fe700 1 --2- 172.21.15.7:0/3350711849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc10aa40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0098a0 tx=0x7f17ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c267277afb08b28 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.979+0000 7f17fa7fc700 1 -- 172.21.15.7:0/3350711849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17ec004160 con 0x7f17fc108650 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.979+0000 7f17fa7fc700 1 -- 172.21.15.7:0/3350711849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17ec0042c0 con 0x7f17fc108650 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.979+0000 7f17fa7fc700 1 -- 172.21.15.7:0/3350711849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17ec011ac0 con 0x7f17fc108650 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.980+0000 7f1801a67700 1 -- 172.21.15.7:0/3350711849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 msgr2=0x7f17fc10aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.980+0000 7f1801a67700 1 --2- 172.21.15.7:0/3350711849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc10aa40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0098a0 tx=0x7f17ec009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.980+0000 7f1801a67700 1 -- 172.21.15.7:0/3350711849 shutdown_connections 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.980+0000 7f1801a67700 1 --2- 172.21.15.7:0/3350711849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc10aa40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.980+0000 7f1801a67700 1 -- 172.21.15.7:0/3350711849 >> 172.21.15.7:0/3350711849 conn(0x7f17fc102040 msgr2=0x7f17fc104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.980+0000 7f1801a67700 1 -- 172.21.15.7:0/3350711849 shutdown_connections 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.980+0000 7f1801a67700 1 -- 172.21.15.7:0/3350711849 wait complete. 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.981+0000 7f1801a67700 1 Processor -- start 2024-03-23T13:10:21.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.981+0000 7f1801a67700 1 -- start start 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.981+0000 7f1801a67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.981+0000 7f1801a67700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc19be60 con 0x7f17fc108650 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.981+0000 7f17fb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.981+0000 7f17fb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc19b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51090/0 (socket says 172.21.15.7:51090) 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.981+0000 7f17fb7fe700 1 -- 172.21.15.7:0/2007257218 learned_addr learned my addr 172.21.15.7:0/2007257218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.981+0000 7f17fb7fe700 1 -- 172.21.15.7:0/2007257218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ec009550 con 0x7f17fc108650 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.982+0000 7f17fb7fe700 1 --2- 172.21.15.7:0/2007257218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc19b920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009130 tx=0x7f17ec011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.982+0000 7f17f8ff9700 1 -- 172.21.15.7:0/2007257218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17ec003aa0 con 0x7f17fc108650 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.982+0000 7f17f8ff9700 1 -- 172.21.15.7:0/2007257218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17ec003c00 con 0x7f17fc108650 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.982+0000 7f1801a67700 1 -- 172.21.15.7:0/2007257218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc19c060 con 0x7f17fc108650 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.982+0000 7f17f8ff9700 1 -- 172.21.15.7:0/2007257218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17ec024800 con 0x7f17fc108650 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.982+0000 7f1801a67700 1 -- 172.21.15.7:0/2007257218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc19c500 con 0x7f17fc108650 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.983+0000 7f17f8ff9700 1 -- 172.21.15.7:0/2007257218 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f17ec024ab0 con 0x7f17fc108650 2024-03-23T13:10:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.983+0000 7f17f8ff9700 1 --2- 172.21.15.7:0/2007257218 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f17e4038f30 0x7f17e403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.983+0000 7f1801a67700 1 -- 172.21.15.7:0/2007257218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17fc10bd30 con 0x7f17fc108650 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.983+0000 7f17f8ff9700 1 -- 172.21.15.7:0/2007257218 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f17ec04d220 con 0x7f17fc108650 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.983+0000 7f17faffd700 1 --2- 172.21.15.7:0/2007257218 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f17e4038f30 0x7f17e403b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.983+0000 7f17faffd700 1 --2- 172.21.15.7:0/2007257218 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f17e4038f30 0x7f17e403b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f17f0006fd0 tx=0x7f17f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:20.988+0000 7f17f8ff9700 1 -- 172.21.15.7:0/2007257218 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f17ec024d60 con 0x7f17fc108650 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.194+0000 7f1801a67700 1 -- 172.21.15.7:0/2007257218 --> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f17fc103210 con 0x7f17e4038f30 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.195+0000 7f17f8ff9700 1 -- 172.21.15.7:0/2007257218 <== mgr.14120 v2:172.21.15.7:6800/3945637763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f17fc103210 con 0x7f17e4038f30 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.197+0000 7f1801a67700 1 -- 172.21.15.7:0/2007257218 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f17e4038f30 msgr2=0x7f17e403b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.197+0000 7f1801a67700 1 --2- 172.21.15.7:0/2007257218 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f17e4038f30 0x7f17e403b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f17f0006fd0 tx=0x7f17f0006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.197+0000 7f1801a67700 1 -- 172.21.15.7:0/2007257218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 msgr2=0x7f17fc19b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.197+0000 7f1801a67700 1 --2- 172.21.15.7:0/2007257218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc19b920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009130 tx=0x7f17ec011b40 comp rx=0 tx=0).stop 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.197+0000 7f1801a67700 1 -- 172.21.15.7:0/2007257218 shutdown_connections 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.197+0000 7f1801a67700 1 --2- 172.21.15.7:0/2007257218 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f17e4038f30 0x7f17e403b3f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.197+0000 7f1801a67700 1 --2- 172.21.15.7:0/2007257218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17fc108650 0x7f17fc19b920 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:21.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.197+0000 7f1801a67700 1 -- 172.21.15.7:0/2007257218 >> 172.21.15.7:0/2007257218 conn(0x7f17fc102040 msgr2=0x7f17fc102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:21.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.197+0000 7f1801a67700 1 -- 172.21.15.7:0/2007257218 shutdown_connections 2024-03-23T13:10:21.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:21.197+0000 7f1801a67700 1 -- 172.21.15.7:0/2007257218 wait complete. 2024-03-23T13:10:21.752 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-23T13:10:21.752 INFO:teuthology.orchestra.run.smithi007.stdout:Adding key to root@localhost authorized_keys... 2024-03-23T13:10:21.752 INFO:teuthology.orchestra.run.smithi007.stdout:Adding host smithi007... 2024-03-23T13:10:22.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:22 smithi007 ceph-mon[32184]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:10:23.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:23 smithi007 ceph-mon[32184]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:10:23.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:23 smithi007 ceph-mon[32184]: Deploying cephadm binary to smithi007 2024-03-23T13:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Added host 'smithi007' with addr '172.21.15.7' 2024-03-23T13:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.187+0000 7f4293551700 1 Processor -- start 2024-03-23T13:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.187+0000 7f4293551700 1 -- start start 2024-03-23T13:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.187+0000 7f4293551700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c108000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.187+0000 7f4293551700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1085d0 con 0x7f428c107be0 2024-03-23T13:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.187+0000 7f42912ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.187+0000 7f42912ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c108000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51098/0 (socket says 172.21.15.7:51098) 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.187+0000 7f42912ed700 1 -- 172.21.15.7:0/2511184212 learned_addr learned my addr 172.21.15.7:0/2511184212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.188+0000 7f42912ed700 1 -- 172.21.15.7:0/2511184212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c108d40 con 0x7f428c107be0 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.188+0000 7f42912ed700 1 --2- 172.21.15.7:0/2511184212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c108000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42880098a0 tx=0x7f4288009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bd58512ef052a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.188+0000 7f4283fff700 1 -- 172.21.15.7:0/2511184212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4288004160 con 0x7f428c107be0 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.188+0000 7f4283fff700 1 -- 172.21.15.7:0/2511184212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42880042c0 con 0x7f428c107be0 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.188+0000 7f4283fff700 1 -- 172.21.15.7:0/2511184212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4288011ac0 con 0x7f428c107be0 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.188+0000 7f4293551700 1 -- 172.21.15.7:0/2511184212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 msgr2=0x7f428c108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.188+0000 7f4293551700 1 --2- 172.21.15.7:0/2511184212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c108000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42880098a0 tx=0x7f4288009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.188+0000 7f4293551700 1 -- 172.21.15.7:0/2511184212 shutdown_connections 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.188+0000 7f4293551700 1 --2- 172.21.15.7:0/2511184212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c108000 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.188+0000 7f4293551700 1 -- 172.21.15.7:0/2511184212 >> 172.21.15.7:0/2511184212 conn(0x7f428c103200 msgr2=0x7f428c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.189+0000 7f4293551700 1 -- 172.21.15.7:0/2511184212 shutdown_connections 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.189+0000 7f4293551700 1 -- 172.21.15.7:0/2511184212 wait complete. 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.189+0000 7f4293551700 1 Processor -- start 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.189+0000 7f4293551700 1 -- start start 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.189+0000 7f4293551700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.189+0000 7f4293551700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f428c19be40 con 0x7f428c107be0 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.190+0000 7f42912ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.190+0000 7f42912ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51100/0 (socket says 172.21.15.7:51100) 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.190+0000 7f42912ed700 1 -- 172.21.15.7:0/2861574919 learned_addr learned my addr 172.21.15.7:0/2861574919 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.190+0000 7f42912ed700 1 -- 172.21.15.7:0/2861574919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4288009550 con 0x7f428c107be0 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.190+0000 7f42912ed700 1 --2- 172.21.15.7:0/2861574919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c19b900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4288000c00 tx=0x7f4288011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.190+0000 7f42827fc700 1 -- 172.21.15.7:0/2861574919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4288003aa0 con 0x7f428c107be0 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.190+0000 7f42827fc700 1 -- 172.21.15.7:0/2861574919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4288003c00 con 0x7f428c107be0 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.190+0000 7f4293551700 1 -- 172.21.15.7:0/2861574919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c19c040 con 0x7f428c107be0 2024-03-23T13:10:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.190+0000 7f42827fc700 1 -- 172.21.15.7:0/2861574919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42880248a0 con 0x7f428c107be0 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.190+0000 7f4293551700 1 -- 172.21.15.7:0/2861574919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c19c4e0 con 0x7f428c107be0 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.191+0000 7f42827fc700 1 -- 172.21.15.7:0/2861574919 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f428801ac60 con 0x7f428c107be0 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.191+0000 7f4293551700 1 -- 172.21.15.7:0/2861574919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c10bf70 con 0x7f428c107be0 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.191+0000 7f42827fc700 1 --2- 172.21.15.7:0/2861574919 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f4278038ee0 0x7f427803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.191+0000 7f42827fc700 1 -- 172.21.15.7:0/2861574919 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f428804d0c0 con 0x7f428c107be0 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.191+0000 7f4290aec700 1 --2- 172.21.15.7:0/2861574919 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f4278038ee0 0x7f427803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.192+0000 7f4290aec700 1 --2- 172.21.15.7:0/2861574919 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f4278038ee0 0x7f427803b3a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f427c006fd0 tx=0x7f427c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.196+0000 7f42827fc700 1 -- 172.21.15.7:0/2861574919 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4288018530 con 0x7f428c107be0 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:22.406+0000 7f4293551700 1 -- 172.21.15.7:0/2861574919 --> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}) -- 0x7f428c104440 con 0x7f4278038ee0 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.720+0000 7f42827fc700 1 -- 172.21.15.7:0/2861574919 <== mgr.14120 v2:172.21.15.7:6800/3945637763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f428c104440 con 0x7f4278038ee0 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.722+0000 7f4293551700 1 -- 172.21.15.7:0/2861574919 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f4278038ee0 msgr2=0x7f427803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.722+0000 7f4293551700 1 --2- 172.21.15.7:0/2861574919 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f4278038ee0 0x7f427803b3a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f427c006fd0 tx=0x7f427c006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.722+0000 7f4293551700 1 -- 172.21.15.7:0/2861574919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 msgr2=0x7f428c19b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.722+0000 7f4293551700 1 --2- 172.21.15.7:0/2861574919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c19b900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4288000c00 tx=0x7f4288011b40 comp rx=0 tx=0).stop 2024-03-23T13:10:27.326 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.722+0000 7f4293551700 1 -- 172.21.15.7:0/2861574919 shutdown_connections 2024-03-23T13:10:27.326 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.722+0000 7f4293551700 1 --2- 172.21.15.7:0/2861574919 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f4278038ee0 0x7f427803b3a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:27.326 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.722+0000 7f4293551700 1 --2- 172.21.15.7:0/2861574919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c107be0 0x7f428c19b900 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:27.326 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.722+0000 7f4293551700 1 -- 172.21.15.7:0/2861574919 >> 172.21.15.7:0/2861574919 conn(0x7f428c103200 msgr2=0x7f428c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:27.326 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.722+0000 7f4293551700 1 -- 172.21.15.7:0/2861574919 shutdown_connections 2024-03-23T13:10:27.326 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:26.722+0000 7f4293551700 1 -- 172.21.15.7:0/2861574919 wait complete. 2024-03-23T13:10:27.326 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mon service... 2024-03-23T13:10:27.876 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:27 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:27.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:27 smithi007 ceph-mon[32184]: Added host smithi007 2024-03-23T13:10:27.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:27 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:29.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:29 smithi007 ceph-mon[32184]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:10:29.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:29 smithi007 ceph-mon[32184]: Saving service mon spec with placement count:5 2024-03-23T13:10:29.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:29 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:29.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-23T13:10:29.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.855+0000 7f638957b700 1 Processor -- start 2024-03-23T13:10:29.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.855+0000 7f638957b700 1 -- start start 2024-03-23T13:10:29.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.855+0000 7f638957b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638407c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.855+0000 7f638957b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f638407c850 con 0x7f638407de10 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.855+0000 7f6382ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638407c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.855+0000 7f6382ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638407c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57044/0 (socket says 172.21.15.7:57044) 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.855+0000 7f6382ffd700 1 -- 172.21.15.7:0/1289581555 learned_addr learned my addr 172.21.15.7:0/1289581555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.855+0000 7f6382ffd700 1 -- 172.21.15.7:0/1289581555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638407c990 con 0x7f638407de10 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.855+0000 7f6382ffd700 1 --2- 172.21.15.7:0/1289581555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638407c310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f63740098a0 tx=0x7f6374009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c1f5cb56db46f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.856+0000 7f6381ffb700 1 -- 172.21.15.7:0/1289581555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6374004160 con 0x7f638407de10 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.856+0000 7f6381ffb700 1 -- 172.21.15.7:0/1289581555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63740042c0 con 0x7f638407de10 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.856+0000 7f6381ffb700 1 -- 172.21.15.7:0/1289581555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6374011ac0 con 0x7f638407de10 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.856+0000 7f638957b700 1 -- 172.21.15.7:0/1289581555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 msgr2=0x7f638407c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.856+0000 7f638957b700 1 --2- 172.21.15.7:0/1289581555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638407c310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f63740098a0 tx=0x7f6374009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.856+0000 7f638957b700 1 -- 172.21.15.7:0/1289581555 shutdown_connections 2024-03-23T13:10:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.856+0000 7f638957b700 1 --2- 172.21.15.7:0/1289581555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638407c310 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.856+0000 7f638957b700 1 -- 172.21.15.7:0/1289581555 >> 172.21.15.7:0/1289581555 conn(0x7f6384103200 msgr2=0x7f6384105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.856+0000 7f638957b700 1 -- 172.21.15.7:0/1289581555 shutdown_connections 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.856+0000 7f638957b700 1 -- 172.21.15.7:0/1289581555 wait complete. 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.857+0000 7f638957b700 1 Processor -- start 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.857+0000 7f638957b700 1 -- start start 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.857+0000 7f638957b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.857+0000 7f638957b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f638419e020 con 0x7f638407de10 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.857+0000 7f6382ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.857+0000 7f6382ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57060/0 (socket says 172.21.15.7:57060) 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.857+0000 7f6382ffd700 1 -- 172.21.15.7:0/3402731550 learned_addr learned my addr 172.21.15.7:0/3402731550 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.858+0000 7f6382ffd700 1 -- 172.21.15.7:0/3402731550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6374009550 con 0x7f638407de10 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.858+0000 7f6382ffd700 1 --2- 172.21.15.7:0/3402731550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638419dae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6374009130 tx=0x7f6374011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.858+0000 7f636bfff700 1 -- 172.21.15.7:0/3402731550 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6374003aa0 con 0x7f638407de10 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.858+0000 7f636bfff700 1 -- 172.21.15.7:0/3402731550 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6374003c00 con 0x7f638407de10 2024-03-23T13:10:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.858+0000 7f638957b700 1 -- 172.21.15.7:0/3402731550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638419e220 con 0x7f638407de10 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.858+0000 7f636bfff700 1 -- 172.21.15.7:0/3402731550 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6374024800 con 0x7f638407de10 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.858+0000 7f638957b700 1 -- 172.21.15.7:0/3402731550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638419e6c0 con 0x7f638407de10 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.859+0000 7f636bfff700 1 -- 172.21.15.7:0/3402731550 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f6374024a70 con 0x7f638407de10 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.859+0000 7f636bfff700 1 --2- 172.21.15.7:0/3402731550 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f636c038f30 0x7f636c03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.859+0000 7f638957b700 1 -- 172.21.15.7:0/3402731550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f638410df30 con 0x7f638407de10 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.859+0000 7f636bfff700 1 -- 172.21.15.7:0/3402731550 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f637404d0a0 con 0x7f638407de10 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.859+0000 7f63827fc700 1 --2- 172.21.15.7:0/3402731550 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f636c038f30 0x7f636c03b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.859+0000 7f63827fc700 1 --2- 172.21.15.7:0/3402731550 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f636c038f30 0x7f636c03b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6378006fd0 tx=0x7f6378006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:27.864+0000 7f636bfff700 1 -- 172.21.15.7:0/3402731550 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f637402d3f0 con 0x7f638407de10 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.091+0000 7f638957b700 1 -- 172.21.15.7:0/3402731550 --> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6384104450 con 0x7f636c038f30 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.095+0000 7f636bfff700 1 -- 172.21.15.7:0/3402731550 <== mgr.14120 v2:172.21.15.7:6800/3945637763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6384104450 con 0x7f636c038f30 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.097+0000 7f638957b700 1 -- 172.21.15.7:0/3402731550 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f636c038f30 msgr2=0x7f636c03b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.097+0000 7f638957b700 1 --2- 172.21.15.7:0/3402731550 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f636c038f30 0x7f636c03b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6378006fd0 tx=0x7f6378006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.097+0000 7f638957b700 1 -- 172.21.15.7:0/3402731550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 msgr2=0x7f638419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.097+0000 7f638957b700 1 --2- 172.21.15.7:0/3402731550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638419dae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6374009130 tx=0x7f6374011b40 comp rx=0 tx=0).stop 2024-03-23T13:10:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.097+0000 7f638957b700 1 -- 172.21.15.7:0/3402731550 shutdown_connections 2024-03-23T13:10:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.097+0000 7f638957b700 1 --2- 172.21.15.7:0/3402731550 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7f636c038f30 0x7f636c03b3f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.097+0000 7f638957b700 1 --2- 172.21.15.7:0/3402731550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638407de10 0x7f638419dae0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.097+0000 7f638957b700 1 -- 172.21.15.7:0/3402731550 >> 172.21.15.7:0/3402731550 conn(0x7f6384103200 msgr2=0x7f6384103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.098+0000 7f638957b700 1 -- 172.21.15.7:0/3402731550 shutdown_connections 2024-03-23T13:10:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:28.098+0000 7f638957b700 1 -- 172.21.15.7:0/3402731550 wait complete. 2024-03-23T13:10:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mgr service... 2024-03-23T13:10:30.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:30 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:30.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:30 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:30.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-23T13:10:30.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.030+0000 7fd4d5ad1700 1 Processor -- start 2024-03-23T13:10:30.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.030+0000 7fd4d5ad1700 1 -- start start 2024-03-23T13:10:30.826 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.030+0000 7fd4d5ad1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d010aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.030+0000 7fd4d5ad1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d007be60 con 0x7fd4d0108610 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.030+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d010aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.030+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d010aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57070/0 (socket says 172.21.15.7:57070) 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.030+0000 7fd4cf7fe700 1 -- 172.21.15.7:0/485086712 learned_addr learned my addr 172.21.15.7:0/485086712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.031+0000 7fd4cf7fe700 1 -- 172.21.15.7:0/485086712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d010af40 con 0x7fd4d0108610 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.031+0000 7fd4cf7fe700 1 --2- 172.21.15.7:0/485086712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d010aa00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098a0 tx=0x7fd4c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3be559dbad70d2d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.031+0000 7fd4ce7fc700 1 -- 172.21.15.7:0/485086712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c0004160 con 0x7fd4d0108610 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.031+0000 7fd4ce7fc700 1 -- 172.21.15.7:0/485086712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4c00042c0 con 0x7fd4d0108610 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.031+0000 7fd4ce7fc700 1 -- 172.21.15.7:0/485086712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c0011a80 con 0x7fd4d0108610 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.032+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/485086712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 msgr2=0x7fd4d010aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.032+0000 7fd4d5ad1700 1 --2- 172.21.15.7:0/485086712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d010aa00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098a0 tx=0x7fd4c0009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.032+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/485086712 shutdown_connections 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.032+0000 7fd4d5ad1700 1 --2- 172.21.15.7:0/485086712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d010aa00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:30.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.032+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/485086712 >> 172.21.15.7:0/485086712 conn(0x7fd4d0102040 msgr2=0x7fd4d0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.032+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/485086712 shutdown_connections 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.032+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/485086712 wait complete. 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.032+0000 7fd4d5ad1700 1 Processor -- start 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4d5ad1700 1 -- start start 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4d5ad1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4d5ad1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d019bdc0 con 0x7fd4d0108610 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57074/0 (socket says 172.21.15.7:57074) 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4cf7fe700 1 -- 172.21.15.7:0/3345100865 learned_addr learned my addr 172.21.15.7:0/3345100865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4cf7fe700 1 -- 172.21.15.7:0/3345100865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c0009550 con 0x7fd4d0108610 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4cf7fe700 1 --2- 172.21.15.7:0/3345100865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d019b880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0011740 tx=0x7fd4c0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4ccff9700 1 -- 172.21.15.7:0/3345100865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c0003b20 con 0x7fd4d0108610 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4ccff9700 1 -- 172.21.15.7:0/3345100865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4c001a910 con 0x7fd4d0108610 2024-03-23T13:10:30.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.033+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/3345100865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d019bfc0 con 0x7fd4d0108610 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.034+0000 7fd4ccff9700 1 -- 172.21.15.7:0/3345100865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c0024900 con 0x7fd4d0108610 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.034+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/3345100865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d019c460 con 0x7fd4d0108610 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.034+0000 7fd4ccff9700 1 -- 172.21.15.7:0/3345100865 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7fd4c0024b40 con 0x7fd4d0108610 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.034+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/3345100865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d010bd80 con 0x7fd4d0108610 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.035+0000 7fd4ccff9700 1 --2- 172.21.15.7:0/3345100865 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd4b8038ee0 0x7fd4b803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.035+0000 7fd4ccff9700 1 -- 172.21.15.7:0/3345100865 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd4c004d490 con 0x7fd4d0108610 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.035+0000 7fd4ceffd700 1 --2- 172.21.15.7:0/3345100865 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd4b8038ee0 0x7fd4b803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.035+0000 7fd4ceffd700 1 --2- 172.21.15.7:0/3345100865 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd4b8038ee0 0x7fd4b803b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4006fd0 tx=0x7fd4c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.040+0000 7fd4ccff9700 1 -- 172.21.15.7:0/3345100865 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd4c0024df0 con 0x7fd4d0108610 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.256+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/3345100865 --> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd4d01031a0 con 0x7fd4b8038ee0 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.260+0000 7fd4ccff9700 1 -- 172.21.15.7:0/3345100865 <== mgr.14120 v2:172.21.15.7:6800/3945637763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd4d01031a0 con 0x7fd4b8038ee0 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.262+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/3345100865 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd4b8038ee0 msgr2=0x7fd4b803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.262+0000 7fd4d5ad1700 1 --2- 172.21.15.7:0/3345100865 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd4b8038ee0 0x7fd4b803b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4006fd0 tx=0x7fd4c4006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:30.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.262+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/3345100865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 msgr2=0x7fd4d019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:30.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.262+0000 7fd4d5ad1700 1 --2- 172.21.15.7:0/3345100865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d019b880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0011740 tx=0x7fd4c0011a50 comp rx=0 tx=0).stop 2024-03-23T13:10:30.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.262+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/3345100865 shutdown_connections 2024-03-23T13:10:30.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.262+0000 7fd4d5ad1700 1 --2- 172.21.15.7:0/3345100865 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd4b8038ee0 0x7fd4b803b3a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:30.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.262+0000 7fd4d5ad1700 1 --2- 172.21.15.7:0/3345100865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0108610 0x7fd4d019b880 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:30.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.262+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/3345100865 >> 172.21.15.7:0/3345100865 conn(0x7fd4d0102040 msgr2=0x7fd4d0102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:30.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.262+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/3345100865 shutdown_connections 2024-03-23T13:10:30.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:30.262+0000 7fd4d5ad1700 1 -- 172.21.15.7:0/3345100865 wait complete. 2024-03-23T13:10:31.442 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:31 smithi007 ceph-mon[32184]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:10:31.442 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:31 smithi007 ceph-mon[32184]: Saving service mgr spec with placement count:2 2024-03-23T13:10:31.442 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:31 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:32.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.224+0000 7fec4b173700 1 Processor -- start 2024-03-23T13:10:32.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.224+0000 7fec4b173700 1 -- start start 2024-03-23T13:10:32.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.225+0000 7fec4b173700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec44107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:32.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.225+0000 7fec4b173700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec44108590 con 0x7fec44107ba0 2024-03-23T13:10:32.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.225+0000 7fec48f0f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec44107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:32.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.225+0000 7fec48f0f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec44107fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57082/0 (socket says 172.21.15.7:57082) 2024-03-23T13:10:32.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.225+0000 7fec48f0f700 1 -- 172.21.15.7:0/1918529585 learned_addr learned my addr 172.21.15.7:0/1918529585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:32.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.225+0000 7fec48f0f700 1 -- 172.21.15.7:0/1918529585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec44108d50 con 0x7fec44107ba0 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.225+0000 7fec48f0f700 1 --2- 172.21.15.7:0/1918529585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec44107fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fec34009e80 tx=0x7fec3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3257c58df7777bfc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.225+0000 7fec437fe700 1 -- 172.21.15.7:0/1918529585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec3400e070 con 0x7fec44107ba0 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.226+0000 7fec437fe700 1 -- 172.21.15.7:0/1918529585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3400b8c0 con 0x7fec44107ba0 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.226+0000 7fec437fe700 1 -- 172.21.15.7:0/1918529585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec340044a0 con 0x7fec44107ba0 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.226+0000 7fec4b173700 1 -- 172.21.15.7:0/1918529585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 msgr2=0x7fec44107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.226+0000 7fec4b173700 1 --2- 172.21.15.7:0/1918529585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec44107fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fec34009e80 tx=0x7fec3400b330 comp rx=0 tx=0).stop 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.226+0000 7fec4b173700 1 -- 172.21.15.7:0/1918529585 shutdown_connections 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.226+0000 7fec4b173700 1 --2- 172.21.15.7:0/1918529585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec44107fc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.226+0000 7fec4b173700 1 -- 172.21.15.7:0/1918529585 >> 172.21.15.7:0/1918529585 conn(0x7fec44103160 msgr2=0x7fec44105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.226+0000 7fec4b173700 1 -- 172.21.15.7:0/1918529585 shutdown_connections 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.226+0000 7fec4b173700 1 -- 172.21.15.7:0/1918529585 wait complete. 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.227+0000 7fec4b173700 1 Processor -- start 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.227+0000 7fec4b173700 1 -- start start 2024-03-23T13:10:32.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.227+0000 7fec4b173700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec4419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.227+0000 7fec4b173700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec4419be40 con 0x7fec44107ba0 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.227+0000 7fec48f0f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec4419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.227+0000 7fec48f0f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec4419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57092/0 (socket says 172.21.15.7:57092) 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.227+0000 7fec48f0f700 1 -- 172.21.15.7:0/1462428762 learned_addr learned my addr 172.21.15.7:0/1462428762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.227+0000 7fec48f0f700 1 -- 172.21.15.7:0/1462428762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec34009b30 con 0x7fec44107ba0 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.228+0000 7fec48f0f700 1 --2- 172.21.15.7:0/1462428762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec4419b900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec34004070 tx=0x7fec3400bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.228+0000 7fec41ffb700 1 -- 172.21.15.7:0/1462428762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec3400e040 con 0x7fec44107ba0 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.228+0000 7fec4b173700 1 -- 172.21.15.7:0/1462428762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4419c040 con 0x7fec44107ba0 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.228+0000 7fec41ffb700 1 -- 172.21.15.7:0/1462428762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec34003d60 con 0x7fec44107ba0 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.228+0000 7fec41ffb700 1 -- 172.21.15.7:0/1462428762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec3401c740 con 0x7fec44107ba0 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.228+0000 7fec4b173700 1 -- 172.21.15.7:0/1462428762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4419c540 con 0x7fec44107ba0 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.229+0000 7fec41ffb700 1 -- 172.21.15.7:0/1462428762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7fec3401c9b0 con 0x7fec44107ba0 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.229+0000 7fec41ffb700 1 --2- 172.21.15.7:0/1462428762 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fec30038f30 0x7fec3003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.229+0000 7fec41ffb700 1 -- 172.21.15.7:0/1462428762 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fec3404ce00 con 0x7fec44107ba0 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.229+0000 7fec43fff700 1 --2- 172.21.15.7:0/1462428762 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fec30038f30 0x7fec3003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.229+0000 7fec4b173700 1 -- 172.21.15.7:0/1462428762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4410bf80 con 0x7fec44107ba0 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.230+0000 7fec43fff700 1 --2- 172.21.15.7:0/1462428762 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fec30038f30 0x7fec3003b3f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec2c006fd0 tx=0x7fec2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.236+0000 7fec41ffb700 1 -- 172.21.15.7:0/1462428762 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fec34017070 con 0x7fec44107ba0 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.451+0000 7fec4b173700 1 -- 172.21.15.7:0/1462428762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fec4410c190 con 0x7fec44107ba0 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.454+0000 7fec41ffb700 1 -- 172.21.15.7:0/1462428762 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fec34020030 con 0x7fec44107ba0 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.456+0000 7fec4b173700 1 -- 172.21.15.7:0/1462428762 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fec30038f30 msgr2=0x7fec3003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.456+0000 7fec4b173700 1 --2- 172.21.15.7:0/1462428762 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fec30038f30 0x7fec3003b3f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec2c006fd0 tx=0x7fec2c006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.456+0000 7fec4b173700 1 -- 172.21.15.7:0/1462428762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 msgr2=0x7fec4419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.456+0000 7fec4b173700 1 --2- 172.21.15.7:0/1462428762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec4419b900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec34004070 tx=0x7fec3400bd10 comp rx=0 tx=0).stop 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.456+0000 7fec4b173700 1 -- 172.21.15.7:0/1462428762 shutdown_connections 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.456+0000 7fec4b173700 1 --2- 172.21.15.7:0/1462428762 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fec30038f30 0x7fec3003b3f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.456+0000 7fec4b173700 1 --2- 172.21.15.7:0/1462428762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec44107ba0 0x7fec4419b900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:32.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.456+0000 7fec4b173700 1 -- 172.21.15.7:0/1462428762 >> 172.21.15.7:0/1462428762 conn(0x7fec44103160 msgr2=0x7fec44103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:32.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.456+0000 7fec4b173700 1 -- 172.21.15.7:0/1462428762 shutdown_connections 2024-03-23T13:10:32.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:31.456+0000 7fec4b173700 1 -- 172.21.15.7:0/1462428762 wait complete. 2024-03-23T13:10:32.751 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:32 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1462428762' entity='client.admin' 2024-03-23T13:10:33.468 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.546+0000 7fdfe0441700 1 Processor -- start 2024-03-23T13:10:33.468 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.546+0000 7fdfe0441700 1 -- start start 2024-03-23T13:10:33.468 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.546+0000 7fdfe0441700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd8106e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:33.468 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.546+0000 7fdfe0441700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd8107450 con 0x7fdfd8106a60 2024-03-23T13:10:33.468 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.547+0000 7fdfde1dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd8106e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:33.468 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.547+0000 7fdfde1dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd8106e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57104/0 (socket says 172.21.15.7:57104) 2024-03-23T13:10:33.468 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.547+0000 7fdfde1dd700 1 -- 172.21.15.7:0/401435419 learned_addr learned my addr 172.21.15.7:0/401435419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:33.468 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.547+0000 7fdfde1dd700 1 -- 172.21.15.7:0/401435419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd8107c60 con 0x7fdfd8106a60 2024-03-23T13:10:33.468 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.547+0000 7fdfde1dd700 1 --2- 172.21.15.7:0/401435419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd8106e80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400bb40 tx=0x7fdfd400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8caab92c064ee4c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.547+0000 7fdfdd1db700 1 -- 172.21.15.7:0/401435419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfd400dbe0 con 0x7fdfd8106a60 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.547+0000 7fdfdd1db700 1 -- 172.21.15.7:0/401435419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd4004510 con 0x7fdfd8106a60 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.547+0000 7fdfdd1db700 1 -- 172.21.15.7:0/401435419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfd4010470 con 0x7fdfd8106a60 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.548+0000 7fdfe0441700 1 -- 172.21.15.7:0/401435419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 msgr2=0x7fdfd8106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.548+0000 7fdfe0441700 1 --2- 172.21.15.7:0/401435419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd8106e80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400bb40 tx=0x7fdfd400be50 comp rx=0 tx=0).stop 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.548+0000 7fdfe0441700 1 -- 172.21.15.7:0/401435419 shutdown_connections 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.548+0000 7fdfe0441700 1 --2- 172.21.15.7:0/401435419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd8106e80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.548+0000 7fdfe0441700 1 -- 172.21.15.7:0/401435419 >> 172.21.15.7:0/401435419 conn(0x7fdfd8102040 msgr2=0x7fdfd8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.548+0000 7fdfe0441700 1 -- 172.21.15.7:0/401435419 shutdown_connections 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.548+0000 7fdfe0441700 1 -- 172.21.15.7:0/401435419 wait complete. 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.549+0000 7fdfe0441700 1 Processor -- start 2024-03-23T13:10:33.469 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.549+0000 7fdfe0441700 1 -- start start 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.549+0000 7fdfe0441700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.549+0000 7fdfe0441700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd819e0f0 con 0x7fdfd8106a60 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.549+0000 7fdfde1dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.549+0000 7fdfde1dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd819dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57114/0 (socket says 172.21.15.7:57114) 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.549+0000 7fdfde1dd700 1 -- 172.21.15.7:0/2868966335 learned_addr learned my addr 172.21.15.7:0/2868966335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.549+0000 7fdfde1dd700 1 -- 172.21.15.7:0/2868966335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd400b7f0 con 0x7fdfd8106a60 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.549+0000 7fdfde1dd700 1 --2- 172.21.15.7:0/2868966335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd819dbb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4003c60 tx=0x7fdfd4003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.549+0000 7fdfcf7fe700 1 -- 172.21.15.7:0/2868966335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfd4004340 con 0x7fdfd8106a60 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.550+0000 7fdfe0441700 1 -- 172.21.15.7:0/2868966335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd819e2f0 con 0x7fdfd8106a60 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.550+0000 7fdfcf7fe700 1 -- 172.21.15.7:0/2868966335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd4010bc0 con 0x7fdfd8106a60 2024-03-23T13:10:33.470 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.550+0000 7fdfcf7fe700 1 -- 172.21.15.7:0/2868966335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfd4019c00 con 0x7fdfd8106a60 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.550+0000 7fdfe0441700 1 -- 172.21.15.7:0/2868966335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd819e730 con 0x7fdfd8106a60 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.551+0000 7fdfcf7fe700 1 -- 172.21.15.7:0/2868966335 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7fdfd40106e0 con 0x7fdfd8106a60 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.551+0000 7fdfe0441700 1 -- 172.21.15.7:0/2868966335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd8197700 con 0x7fdfd8106a60 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.551+0000 7fdfcf7fe700 1 --2- 172.21.15.7:0/2868966335 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fdfc4038ee0 0x7fdfc403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.551+0000 7fdfcf7fe700 1 -- 172.21.15.7:0/2868966335 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdfd404d1f0 con 0x7fdfd8106a60 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.551+0000 7fdfdd9dc700 1 --2- 172.21.15.7:0/2868966335 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fdfc4038ee0 0x7fdfc403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.551+0000 7fdfdd9dc700 1 --2- 172.21.15.7:0/2868966335 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fdfc4038ee0 0x7fdfc403b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8006fd0 tx=0x7fdfc8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.556+0000 7fdfcf7fe700 1 -- 172.21.15.7:0/2868966335 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fdfd4006d80 con 0x7fdfd8106a60 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.759+0000 7fdfe0441700 1 -- 172.21.15.7:0/2868966335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fdfd807afc0 con 0x7fdfd8106a60 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.762+0000 7fdfcf7fe700 1 -- 172.21.15.7:0/2868966335 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fdfd400f3d0 con 0x7fdfd8106a60 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.764+0000 7fdfe0441700 1 -- 172.21.15.7:0/2868966335 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fdfc4038ee0 msgr2=0x7fdfc403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:33.471 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.764+0000 7fdfe0441700 1 --2- 172.21.15.7:0/2868966335 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fdfc4038ee0 0x7fdfc403b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8006fd0 tx=0x7fdfc8006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:33.472 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.764+0000 7fdfe0441700 1 -- 172.21.15.7:0/2868966335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 msgr2=0x7fdfd819dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:33.472 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.764+0000 7fdfe0441700 1 --2- 172.21.15.7:0/2868966335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd819dbb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4003c60 tx=0x7fdfd4003980 comp rx=0 tx=0).stop 2024-03-23T13:10:33.472 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.764+0000 7fdfe0441700 1 -- 172.21.15.7:0/2868966335 shutdown_connections 2024-03-23T13:10:33.472 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.764+0000 7fdfe0441700 1 --2- 172.21.15.7:0/2868966335 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fdfc4038ee0 0x7fdfc403b3a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:33.472 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.764+0000 7fdfe0441700 1 --2- 172.21.15.7:0/2868966335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd8106a60 0x7fdfd819dbb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:33.472 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.764+0000 7fdfe0441700 1 -- 172.21.15.7:0/2868966335 >> 172.21.15.7:0/2868966335 conn(0x7fdfd8102040 msgr2=0x7fdfd8194840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:33.472 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.764+0000 7fdfe0441700 1 -- 172.21.15.7:0/2868966335 shutdown_connections 2024-03-23T13:10:33.472 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:32.764+0000 7fdfe0441700 1 -- 172.21.15.7:0/2868966335 wait complete. 2024-03-23T13:10:33.472 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling the dashboard module... 2024-03-23T13:10:34.025 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:33 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2868966335' entity='client.admin' 2024-03-23T13:10:34.025 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:33 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:35.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:34 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4040562414' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-23T13:10:35.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:34 smithi007 ceph-mon[32184]: from='mgr.14120 172.21.15.7:0/2599323522' entity='mgr.x' 2024-03-23T13:10:35.067 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: ignoring --setuser ceph since I am not root 2024-03-23T13:10:35.067 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: ignoring --setgroup ceph since I am not root 2024-03-23T13:10:35.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.873+0000 7ff5c2bf6700 1 Processor -- start 2024-03-23T13:10:35.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.873+0000 7ff5c2bf6700 1 -- start start 2024-03-23T13:10:35.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.873+0000 7ff5c2bf6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:35.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.873+0000 7ff5c2bf6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc108590 con 0x7ff5bc107ba0 2024-03-23T13:10:35.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.873+0000 7ff5c0992700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:35.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.874+0000 7ff5c0992700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc107fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57116/0 (socket says 172.21.15.7:57116) 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.874+0000 7ff5c0992700 1 -- 172.21.15.7:0/3513459315 learned_addr learned my addr 172.21.15.7:0/3513459315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.874+0000 7ff5c0992700 1 -- 172.21.15.7:0/3513459315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc108d50 con 0x7ff5bc107ba0 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.874+0000 7ff5c0992700 1 --2- 172.21.15.7:0/3513459315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc107fc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009e80 tx=0x7ff5ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fe2268d315590 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.874+0000 7ff5bb7fe700 1 -- 172.21.15.7:0/3513459315 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ac00e070 con 0x7ff5bc107ba0 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.874+0000 7ff5bb7fe700 1 -- 172.21.15.7:0/3513459315 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ac00b8c0 con 0x7ff5bc107ba0 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.874+0000 7ff5bb7fe700 1 -- 172.21.15.7:0/3513459315 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ac0044a0 con 0x7ff5bc107ba0 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.875+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/3513459315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 msgr2=0x7ff5bc107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.875+0000 7ff5c2bf6700 1 --2- 172.21.15.7:0/3513459315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc107fc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009e80 tx=0x7ff5ac00b330 comp rx=0 tx=0).stop 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.875+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/3513459315 shutdown_connections 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.875+0000 7ff5c2bf6700 1 --2- 172.21.15.7:0/3513459315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc107fc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.875+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/3513459315 >> 172.21.15.7:0/3513459315 conn(0x7ff5bc103160 msgr2=0x7ff5bc105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.875+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/3513459315 shutdown_connections 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.875+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/3513459315 wait complete. 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.875+0000 7ff5c2bf6700 1 Processor -- start 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.875+0000 7ff5c2bf6700 1 -- start start 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5c2bf6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5c2bf6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc19be20 con 0x7ff5bc107ba0 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5c0992700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5c0992700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc19b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57126/0 (socket says 172.21.15.7:57126) 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5c0992700 1 -- 172.21.15.7:0/4040562414 learned_addr learned my addr 172.21.15.7:0/4040562414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5c0992700 1 -- 172.21.15.7:0/4040562414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac009b30 con 0x7ff5bc107ba0 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5c0992700 1 --2- 172.21.15.7:0/4040562414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc19b8e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac004070 tx=0x7ff5ac00bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:35.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5b9ffb700 1 -- 172.21.15.7:0/4040562414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ac00e040 con 0x7ff5bc107ba0 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/4040562414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc19c020 con 0x7ff5bc107ba0 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5b9ffb700 1 -- 172.21.15.7:0/4040562414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ac003d60 con 0x7ff5bc107ba0 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.876+0000 7ff5b9ffb700 1 -- 172.21.15.7:0/4040562414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ac01c740 con 0x7ff5bc107ba0 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.877+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/4040562414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc19c520 con 0x7ff5bc107ba0 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.877+0000 7ff5b9ffb700 1 -- 172.21.15.7:0/4040562414 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7ff5ac01ca00 con 0x7ff5bc107ba0 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.877+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/4040562414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc10bf80 con 0x7ff5bc107ba0 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.878+0000 7ff5b9ffb700 1 --2- 172.21.15.7:0/4040562414 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7ff5a4038ee0 0x7ff5a403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.878+0000 7ff5b9ffb700 1 -- 172.21.15.7:0/4040562414 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff5ac04cdf0 con 0x7ff5bc107ba0 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.878+0000 7ff5bbfff700 1 --2- 172.21.15.7:0/4040562414 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7ff5a4038ee0 0x7ff5a403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.878+0000 7ff5bbfff700 1 --2- 172.21.15.7:0/4040562414 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7ff5a4038ee0 0x7ff5a403b3a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0006fd0 tx=0x7ff5b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:35.370 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:33.883+0000 7ff5b9ffb700 1 -- 172.21.15.7:0/4040562414 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff5ac011980 con 0x7ff5bc107ba0 2024-03-23T13:10:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.147+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/4040562414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff5bc19f200 con 0x7ff5bc107ba0 2024-03-23T13:10:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.764+0000 7ff5b9ffb700 1 -- 172.21.15.7:0/4040562414 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff5ac017070 con 0x7ff5bc107ba0 2024-03-23T13:10:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.765+0000 7ff5b9ffb700 1 -- 172.21.15.7:0/4040562414 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 9) ==== 45941+0+0 (secure 0 0 0) 0x7ff5ac01b580 con 0x7ff5bc107ba0 2024-03-23T13:10:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.767+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/4040562414 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7ff5a4038ee0 msgr2=0x7ff5a403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.767+0000 7ff5c2bf6700 1 --2- 172.21.15.7:0/4040562414 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7ff5a4038ee0 0x7ff5a403b3a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0006fd0 tx=0x7ff5b0006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.767+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/4040562414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 msgr2=0x7ff5bc19b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.767+0000 7ff5c2bf6700 1 --2- 172.21.15.7:0/4040562414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc19b8e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac004070 tx=0x7ff5ac00bd10 comp rx=0 tx=0).stop 2024-03-23T13:10:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.767+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/4040562414 shutdown_connections 2024-03-23T13:10:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.767+0000 7ff5c2bf6700 1 --2- 172.21.15.7:0/4040562414 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7ff5a4038ee0 0x7ff5a403b3a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:35.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.767+0000 7ff5c2bf6700 1 --2- 172.21.15.7:0/4040562414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc107ba0 0x7ff5bc19b8e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:35.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.767+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/4040562414 >> 172.21.15.7:0/4040562414 conn(0x7ff5bc103160 msgr2=0x7ff5bc103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:35.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.767+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/4040562414 shutdown_connections 2024-03-23T13:10:35.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:34.767+0000 7ff5c2bf6700 1 -- 172.21.15.7:0/4040562414 wait complete. 2024-03-23T13:10:35.678 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:35 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:35.353+0000 7fb638d89200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T13:10:35.678 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:35 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:35.458+0000 7fb638d89200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T13:10:35.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:35 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4040562414' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-23T13:10:35.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:35 smithi007 ceph-mon[32184]: mgrmap e9: x(active, since 19s) 2024-03-23T13:10:36.235 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:36.144+0000 7fb638d89200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T13:10:36.236 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:36.244+0000 7fb638d89200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T13:10:36.512 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:36.388+0000 7fb638d89200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T13:10:36.512 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:36.486+0000 7fb638d89200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T13:10:36.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:36 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1283076696' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-23T13:10:36.818 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:36.572+0000 7fb638d89200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.791+0000 7fd3afaac700 1 Processor -- start 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.791+0000 7fd3afaac700 1 -- start start 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.792+0000 7fd3afaac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a8108470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.792+0000 7fd3afaac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8108a40 con 0x7fd3a8108050 2024-03-23T13:10:36.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.792+0000 7fd3ad848700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a8108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.792+0000 7fd3ad848700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a8108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53592/0 (socket says 172.21.15.7:53592) 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.792+0000 7fd3ad848700 1 -- 172.21.15.7:0/620148100 learned_addr learned my addr 172.21.15.7:0/620148100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.793+0000 7fd3ad848700 1 -- 172.21.15.7:0/620148100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a8108b80 con 0x7fd3a8108050 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.793+0000 7fd3ad848700 1 --2- 172.21.15.7:0/620148100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a8108470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40098a0 tx=0x7fd3a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ee7870be91b059 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.793+0000 7fd3ac846700 1 -- 172.21.15.7:0/620148100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a4004160 con 0x7fd3a8108050 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.793+0000 7fd3ac846700 1 -- 172.21.15.7:0/620148100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a40042c0 con 0x7fd3a8108050 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.793+0000 7fd3ac846700 1 -- 172.21.15.7:0/620148100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a4011a80 con 0x7fd3a8108050 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.794+0000 7fd3afaac700 1 -- 172.21.15.7:0/620148100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 msgr2=0x7fd3a8108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.794+0000 7fd3afaac700 1 --2- 172.21.15.7:0/620148100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a8108470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40098a0 tx=0x7fd3a4009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.794+0000 7fd3afaac700 1 -- 172.21.15.7:0/620148100 shutdown_connections 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.794+0000 7fd3afaac700 1 --2- 172.21.15.7:0/620148100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a8108470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.794+0000 7fd3afaac700 1 -- 172.21.15.7:0/620148100 >> 172.21.15.7:0/620148100 conn(0x7fd3a8101820 msgr2=0x7fd3a8103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:36.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.794+0000 7fd3afaac700 1 -- 172.21.15.7:0/620148100 shutdown_connections 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.794+0000 7fd3afaac700 1 -- 172.21.15.7:0/620148100 wait complete. 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.795+0000 7fd3afaac700 1 Processor -- start 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.795+0000 7fd3afaac700 1 -- start start 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.795+0000 7fd3afaac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.795+0000 7fd3afaac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a819e000 con 0x7fd3a8108050 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.795+0000 7fd3ad848700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.795+0000 7fd3ad848700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53604/0 (socket says 172.21.15.7:53604) 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.795+0000 7fd3ad848700 1 -- 172.21.15.7:0/1283076696 learned_addr learned my addr 172.21.15.7:0/1283076696 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.796+0000 7fd3ad848700 1 -- 172.21.15.7:0/1283076696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a4009550 con 0x7fd3a8108050 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.796+0000 7fd3ad848700 1 --2- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a819dac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4011740 tx=0x7fd3a4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.796+0000 7fd39effd700 1 -- 172.21.15.7:0/1283076696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a4003b20 con 0x7fd3a8108050 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.796+0000 7fd39effd700 1 -- 172.21.15.7:0/1283076696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a401a910 con 0x7fd3a8108050 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.796+0000 7fd39effd700 1 -- 172.21.15.7:0/1283076696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a40248c0 con 0x7fd3a8108050 2024-03-23T13:10:36.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.796+0000 7fd3afaac700 1 -- 172.21.15.7:0/1283076696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a819e200 con 0x7fd3a8108050 2024-03-23T13:10:36.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.796+0000 7fd3afaac700 1 -- 172.21.15.7:0/1283076696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a819e6a0 con 0x7fd3a8108050 2024-03-23T13:10:36.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.797+0000 7fd39effd700 1 -- 172.21.15.7:0/1283076696 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 45941+0+0 (secure 0 0 0) 0x7fd3a402d3f0 con 0x7fd3a8108050 2024-03-23T13:10:36.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.797+0000 7fd3afaac700 1 -- 172.21.15.7:0/1283076696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a8106890 con 0x7fd3a8108050 2024-03-23T13:10:36.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.797+0000 7fd39effd700 1 --2- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd394038f90 0x7fd39403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:36.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.798+0000 7fd39effd700 1 -- 172.21.15.7:0/1283076696 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd3a404d340 con 0x7fd3a8108050 2024-03-23T13:10:36.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.798+0000 7fd3ad047700 1 -- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd394038f90 msgr2=0x7fd39403b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3945637763 2024-03-23T13:10:36.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.798+0000 7fd3ad047700 1 --2- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd394038f90 0x7fd39403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T13:10:36.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.803+0000 7fd39effd700 1 -- 172.21.15.7:0/1283076696 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd3a4010e80 con 0x7fd3a8108050 2024-03-23T13:10:36.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.999+0000 7fd3ad047700 1 -- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd394038f90 msgr2=0x7fd39403b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3945637763 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:35.999+0000 7fd3ad047700 1 --2- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd394038f90 0x7fd39403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.077+0000 7fd3afaac700 1 -- 172.21.15.7:0/1283076696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd3a8197190 con 0x7fd3a8108050 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.077+0000 7fd39effd700 1 -- 172.21.15.7:0/1283076696 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fd3a4010380 con 0x7fd3a8108050 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.080+0000 7fd3afaac700 1 -- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd394038f90 msgr2=0x7fd39403b450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.080+0000 7fd3afaac700 1 --2- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd394038f90 0x7fd39403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.080+0000 7fd3afaac700 1 -- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 msgr2=0x7fd3a819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.080+0000 7fd3afaac700 1 --2- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a819dac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4011740 tx=0x7fd3a4011b00 comp rx=0 tx=0).stop 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.080+0000 7fd3afaac700 1 -- 172.21.15.7:0/1283076696 shutdown_connections 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.080+0000 7fd3afaac700 1 --2- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fd394038f90 0x7fd39403b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.080+0000 7fd3afaac700 1 --2- 172.21.15.7:0/1283076696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8108050 0x7fd3a819dac0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.080+0000 7fd3afaac700 1 -- 172.21.15.7:0/1283076696 >> 172.21.15.7:0/1283076696 conn(0x7fd3a8101820 msgr2=0x7fd3a8194270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.080+0000 7fd3afaac700 1 -- 172.21.15.7:0/1283076696 shutdown_connections 2024-03-23T13:10:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:36.080+0000 7fd3afaac700 1 -- 172.21.15.7:0/1283076696 wait complete. 2024-03-23T13:10:36.940 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-03-23T13:10:36.940 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 9... 2024-03-23T13:10:37.138 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:36.843+0000 7fb638d89200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T13:10:37.138 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:36.958+0000 7fb638d89200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T13:10:37.138 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:37 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:37.047+0000 7fb638d89200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T13:10:37.397 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:37 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:37.146+0000 7fb638d89200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T13:10:37.397 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:37 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:37.329+0000 7fb638d89200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T13:10:38.448 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:38.322+0000 7fb638d89200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T13:10:38.448 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:38.456+0000 7fb638d89200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T13:10:39.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:38.795+0000 7fb638d89200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T13:10:39.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:38.898+0000 7fb638d89200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T13:10:39.411 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:39.100+0000 7fb638d89200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T13:10:39.411 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:39.291+0000 7fb638d89200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T13:10:39.818 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:39.589+0000 7fb638d89200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T13:10:40.318 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:39.850+0000 7fb638d89200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T13:10:40.318 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:40 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:40.094+0000 7fb638d89200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T13:10:41.318 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:40 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:40.925+0000 7fb638d89200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T13:10:41.818 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:41.505+0000 7fb638d89200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T13:10:41.818 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:41.606+0000 7fb638d89200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T13:10:42.318 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:10:41.828+0000 7fb638d89200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T13:10:42.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: Active manager daemon x restarted 2024-03-23T13:10:42.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: Activating manager daemon x 2024-03-23T13:10:42.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: osdmap e3: 0 total, 0 up, 0 in 2024-03-23T13:10:42.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: mgrmap e10: x(active, starting, since 0.00329526s) 2024-03-23T13:10:42.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:10:42.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T13:10:42.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T13:10:42.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T13:10:42.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T13:10:42.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: Manager daemon x is now available 2024-03-23T13:10:42.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:42.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:43.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T13:10:43.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T13:10:43.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:42 smithi007 ceph-mon[32184]: mgrmap e11: x(active, since 1.00589s) 2024-03-23T13:10:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-23T13:10:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-23T13:10:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-23T13:10:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-23T13:10:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.405+0000 7fc232686700 1 Processor -- start 2024-03-23T13:10:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.405+0000 7fc232686700 1 -- start start 2024-03-23T13:10:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.405+0000 7fc232686700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.405+0000 7fc232686700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c07c830 con 0x7fc22c07ddf0 2024-03-23T13:10:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.405+0000 7fc22bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.405+0000 7fc22bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53618/0 (socket says 172.21.15.7:53618) 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.405+0000 7fc22bfff700 1 -- 172.21.15.7:0/1199524304 learned_addr learned my addr 172.21.15.7:0/1199524304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.405+0000 7fc22bfff700 1 -- 172.21.15.7:0/1199524304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c07c970 con 0x7fc22c07ddf0 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.406+0000 7fc22bfff700 1 --2- 172.21.15.7:0/1199524304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c07c2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098a0 tx=0x7fc21c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1001e63b9997c9a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.406+0000 7fc22affd700 1 -- 172.21.15.7:0/1199524304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc21c004160 con 0x7fc22c07ddf0 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.406+0000 7fc22affd700 1 -- 172.21.15.7:0/1199524304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc21c0042c0 con 0x7fc22c07ddf0 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.406+0000 7fc22affd700 1 -- 172.21.15.7:0/1199524304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc21c011a80 con 0x7fc22c07ddf0 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.406+0000 7fc232686700 1 -- 172.21.15.7:0/1199524304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 msgr2=0x7fc22c07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.406+0000 7fc232686700 1 --2- 172.21.15.7:0/1199524304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c07c2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098a0 tx=0x7fc21c009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.406+0000 7fc232686700 1 -- 172.21.15.7:0/1199524304 shutdown_connections 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.406+0000 7fc232686700 1 --2- 172.21.15.7:0/1199524304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c07c2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.406+0000 7fc232686700 1 -- 172.21.15.7:0/1199524304 >> 172.21.15.7:0/1199524304 conn(0x7fc22c1031c0 msgr2=0x7fc22c1055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.407+0000 7fc232686700 1 -- 172.21.15.7:0/1199524304 shutdown_connections 2024-03-23T13:10:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.407+0000 7fc232686700 1 -- 172.21.15.7:0/1199524304 wait complete. 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.407+0000 7fc232686700 1 Processor -- start 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.407+0000 7fc232686700 1 -- start start 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc232686700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc232686700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c19dfb0 con 0x7fc22c07ddf0 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc22bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc22bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53628/0 (socket says 172.21.15.7:53628) 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc22bfff700 1 -- 172.21.15.7:0/3540426940 learned_addr learned my addr 172.21.15.7:0/3540426940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc22bfff700 1 -- 172.21.15.7:0/3540426940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c009550 con 0x7fc22c07ddf0 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc22bfff700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c19da70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc21c011740 tx=0x7fc21c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc21c003b20 con 0x7fc22c07ddf0 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc232686700 1 -- 172.21.15.7:0/3540426940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c19e1b0 con 0x7fc22c07ddf0 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc232686700 1 -- 172.21.15.7:0/3540426940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c19e650 con 0x7fc22c07ddf0 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.408+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc21c01a910 con 0x7fc22c07ddf0 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.409+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc21c0248c0 con 0x7fc22c07ddf0 2024-03-23T13:10:43.727 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.409+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 45941+0+0 (secure 0 0 0) 0x7fc21c02d3f0 con 0x7fc22c07ddf0 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.409+0000 7fc2297fa700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 0x7fc21403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.409+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 --> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc21403bb60 con 0x7fc214038f90 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.409+0000 7fc22b7fe700 1 -- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 msgr2=0x7fc21403b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3945637763 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.410+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc21c04d640 con 0x7fc22c07ddf0 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.410+0000 7fc22b7fe700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 0x7fc21403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.611+0000 7fc22b7fe700 1 -- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 msgr2=0x7fc21403b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3945637763 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:37.611+0000 7fc22b7fe700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 0x7fc21403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:38.012+0000 7fc22b7fe700 1 -- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 msgr2=0x7fc21403b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3945637763 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:38.012+0000 7fc22b7fe700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 0x7fc21403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:38.813+0000 7fc22b7fe700 1 -- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 msgr2=0x7fc21403b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3945637763 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:38.813+0000 7fc22b7fe700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 0x7fc21403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:40.414+0000 7fc22b7fe700 1 -- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 msgr2=0x7fc21403b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3945637763 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:40.414+0000 7fc22b7fe700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 0x7fc21403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:41.843+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 10) ==== 45708+0+0 (secure 0 0 0) 0x7fc21c02d920 con 0x7fc22c07ddf0 2024-03-23T13:10:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:41.844+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 msgr2=0x7fc21403b450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:41.844+0000 7fc2297fa700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 0x7fc21403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.845+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 11) ==== 45835+0+0 (secure 0 0 0) 0x7fc21c04ed20 con 0x7fc22c07ddf0 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.845+0000 7fc2297fa700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc21403ca40 0x7fc21403ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.845+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc21403bb60 con 0x7fc21403ca40 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.846+0000 7fc22b7fe700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc21403ca40 0x7fc21403ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.847+0000 7fc22b7fe700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc21403ca40 0x7fc21403ee30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc220003a10 tx=0x7fc220009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.847+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fc21403bb60 con 0x7fc21403ca40 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.854+0000 7fc232686700 1 -- 172.21.15.7:0/3540426940 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc218002630 con 0x7fc21403ca40 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.854+0000 7fc2297fa700 1 -- 172.21.15.7:0/3540426940 <== mgr.14152 v2:172.21.15.7:6800/1535208045 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fc218002630 con 0x7fc21403ca40 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 -- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc21403ca40 msgr2=0x7fc21403ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc21403ca40 0x7fc21403ee30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc220003a10 tx=0x7fc220009270 comp rx=0 tx=0).stop 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 -- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 msgr2=0x7fc22c19da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c19da70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc21c011740 tx=0x7fc21c011b00 comp rx=0 tx=0).stop 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 -- 172.21.15.7:0/3540426940 shutdown_connections 2024-03-23T13:10:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc21403ca40 0x7fc21403ee30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:43.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc22c07ddf0 0x7fc22c19da70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:43.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 --2- 172.21.15.7:0/3540426940 >> [v2:172.21.15.7:6800/3945637763,v1:172.21.15.7:6801/3945637763] conn(0x7fc214038f90 0x7fc21403b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:43.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 -- 172.21.15.7:0/3540426940 >> 172.21.15.7:0/3540426940 conn(0x7fc22c1031c0 msgr2=0x7fc22c103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:43.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 -- 172.21.15.7:0/3540426940 shutdown_connections 2024-03-23T13:10:43.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:42.855+0000 7fc232686700 1 -- 172.21.15.7:0/3540426940 wait complete. 2024-03-23T13:10:43.730 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 9 is available 2024-03-23T13:10:43.730 INFO:teuthology.orchestra.run.smithi007.stdout:Generating a dashboard self-signed certificate... 2024-03-23T13:10:44.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:44 smithi007 ceph-mon[32184]: [23/Mar/2024:13:10:43] ENGINE Bus STARTING 2024-03-23T13:10:44.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:44 smithi007 ceph-mon[32184]: [23/Mar/2024:13:10:43] ENGINE Serving on http://172.21.15.7:8765 2024-03-23T13:10:44.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:44 smithi007 ceph-mon[32184]: [23/Mar/2024:13:10:43] ENGINE Serving on https://172.21.15.7:7150 2024-03-23T13:10:44.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:44 smithi007 ceph-mon[32184]: [23/Mar/2024:13:10:43] ENGINE Bus STARTED 2024-03-23T13:10:44.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:44 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.226+0000 7fde204eb700 1 Processor -- start 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.226+0000 7fde204eb700 1 -- start start 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.226+0000 7fde204eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1810a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.226+0000 7fde204eb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde1807bea0 con 0x7fde181085e0 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.226+0000 7fde1e287700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1810a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.226+0000 7fde1e287700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1810a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53682/0 (socket says 172.21.15.7:53682) 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.226+0000 7fde1e287700 1 -- 172.21.15.7:0/548449798 learned_addr learned my addr 172.21.15.7:0/548449798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.227+0000 7fde1e287700 1 -- 172.21.15.7:0/548449798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1810af10 con 0x7fde181085e0 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.227+0000 7fde1e287700 1 --2- 172.21.15.7:0/548449798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1810a9d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde140098a0 tx=0x7fde14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6118c938c023bfac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:45.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.227+0000 7fde1d285700 1 -- 172.21.15.7:0/548449798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde14004160 con 0x7fde181085e0 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.227+0000 7fde1d285700 1 -- 172.21.15.7:0/548449798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde140042c0 con 0x7fde181085e0 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.227+0000 7fde1d285700 1 -- 172.21.15.7:0/548449798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde14011a80 con 0x7fde181085e0 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.228+0000 7fde204eb700 1 -- 172.21.15.7:0/548449798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 msgr2=0x7fde1810a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.228+0000 7fde204eb700 1 --2- 172.21.15.7:0/548449798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1810a9d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde140098a0 tx=0x7fde14009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.228+0000 7fde204eb700 1 -- 172.21.15.7:0/548449798 shutdown_connections 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.228+0000 7fde204eb700 1 --2- 172.21.15.7:0/548449798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1810a9d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.228+0000 7fde204eb700 1 -- 172.21.15.7:0/548449798 >> 172.21.15.7:0/548449798 conn(0x7fde18102030 msgr2=0x7fde18104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.228+0000 7fde204eb700 1 -- 172.21.15.7:0/548449798 shutdown_connections 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.228+0000 7fde204eb700 1 -- 172.21.15.7:0/548449798 wait complete. 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.228+0000 7fde204eb700 1 Processor -- start 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde204eb700 1 -- start start 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde204eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde204eb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde1819be00 con 0x7fde181085e0 2024-03-23T13:10:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde1e287700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde1e287700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1819b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53696/0 (socket says 172.21.15.7:53696) 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde1e287700 1 -- 172.21.15.7:0/3848202570 learned_addr learned my addr 172.21.15.7:0/3848202570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde1e287700 1 -- 172.21.15.7:0/3848202570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde14009550 con 0x7fde181085e0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde1e287700 1 --2- 172.21.15.7:0/3848202570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1819b8c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fde14011740 tx=0x7fde14011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde0f7fe700 1 -- 172.21.15.7:0/3848202570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde14003b20 con 0x7fde181085e0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde0f7fe700 1 -- 172.21.15.7:0/3848202570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde1401a910 con 0x7fde181085e0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.229+0000 7fde204eb700 1 -- 172.21.15.7:0/3848202570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1819c000 con 0x7fde181085e0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.230+0000 7fde0f7fe700 1 -- 172.21.15.7:0/3848202570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde140248c0 con 0x7fde181085e0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.230+0000 7fde204eb700 1 -- 172.21.15.7:0/3848202570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1819c4a0 con 0x7fde181085e0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.230+0000 7fde0f7fe700 1 -- 172.21.15.7:0/3848202570 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) ==== 45835+0+0 (secure 0 0 0) 0x7fde1402d3f0 con 0x7fde181085e0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.231+0000 7fde0f7fe700 1 --2- 172.21.15.7:0/3848202570 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fde04038e80 0x7fde0403b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.231+0000 7fde0f7fe700 1 -- 172.21.15.7:0/3848202570 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fde1404d4d0 con 0x7fde181085e0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.231+0000 7fde1da86700 1 --2- 172.21.15.7:0/3848202570 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fde04038e80 0x7fde0403b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.231+0000 7fde204eb700 1 -- 172.21.15.7:0/3848202570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1810bdb0 con 0x7fde181085e0 2024-03-23T13:10:45.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.231+0000 7fde1da86700 1 --2- 172.21.15.7:0/3848202570 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fde04038e80 0x7fde0403b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde08006fd0 tx=0x7fde08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.236+0000 7fde0f7fe700 1 -- 172.21.15.7:0/3848202570 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fde1402d740 con 0x7fde181085e0 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.449+0000 7fde204eb700 1 -- 172.21.15.7:0/3848202570 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fde18103200 con 0x7fde04038e80 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.461+0000 7fde0f7fe700 1 -- 172.21.15.7:0/3848202570 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 12) ==== 45941+0+0 (secure 0 0 0) 0x7fde14021d60 con 0x7fde181085e0 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.545+0000 7fde0f7fe700 1 -- 172.21.15.7:0/3848202570 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fde18103200 con 0x7fde04038e80 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.547+0000 7fde204eb700 1 -- 172.21.15.7:0/3848202570 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fde04038e80 msgr2=0x7fde0403b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.547+0000 7fde204eb700 1 --2- 172.21.15.7:0/3848202570 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fde04038e80 0x7fde0403b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde08006fd0 tx=0x7fde08006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.547+0000 7fde204eb700 1 -- 172.21.15.7:0/3848202570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 msgr2=0x7fde1819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.547+0000 7fde204eb700 1 --2- 172.21.15.7:0/3848202570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1819b8c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fde14011740 tx=0x7fde14011b00 comp rx=0 tx=0).stop 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.547+0000 7fde204eb700 1 -- 172.21.15.7:0/3848202570 shutdown_connections 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.547+0000 7fde204eb700 1 --2- 172.21.15.7:0/3848202570 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fde04038e80 0x7fde0403b340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.547+0000 7fde204eb700 1 --2- 172.21.15.7:0/3848202570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde181085e0 0x7fde1819b8c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.548+0000 7fde204eb700 1 -- 172.21.15.7:0/3848202570 >> 172.21.15.7:0/3848202570 conn(0x7fde18102030 msgr2=0x7fde18102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.548+0000 7fde204eb700 1 -- 172.21.15.7:0/3848202570 shutdown_connections 2024-03-23T13:10:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:44.548+0000 7fde204eb700 1 -- 172.21.15.7:0/3848202570 wait complete. 2024-03-23T13:10:45.110 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial admin user... 2024-03-23T13:10:45.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:45 smithi007 ceph-mon[32184]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:10:45.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:45 smithi007 ceph-mon[32184]: mgrmap e12: x(active, since 2s) 2024-03-23T13:10:45.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:45 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:45.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:45 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:45.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:45 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:46.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$JuObnnFYrq0Du8o.4PiFcOWTup.lXyyMwvvrsNK0d4TWHCRliO4cu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711199446, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-23T13:10:46.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.531+0000 7f35ed131700 1 Processor -- start 2024-03-23T13:10:46.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.531+0000 7f35ed131700 1 -- start start 2024-03-23T13:10:46.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.531+0000 7f35ed131700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:46.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.531+0000 7f35ed131700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35e807c810 con 0x7f35e807ddd0 2024-03-23T13:10:46.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.531+0000 7f35e6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:46.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.531+0000 7f35e6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36522/0 (socket says 172.21.15.7:36522) 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.531+0000 7f35e6d9d700 1 -- 172.21.15.7:0/3952822827 learned_addr learned my addr 172.21.15.7:0/3952822827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.532+0000 7f35e6d9d700 1 -- 172.21.15.7:0/3952822827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e807c950 con 0x7f35e807ddd0 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.532+0000 7f35e6d9d700 1 --2- 172.21.15.7:0/3952822827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e807c2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f35d80098a0 tx=0x7f35d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49eb8016e0dc1fd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.532+0000 7f35e5d9b700 1 -- 172.21.15.7:0/3952822827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35d8004160 con 0x7f35e807ddd0 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.532+0000 7f35e5d9b700 1 -- 172.21.15.7:0/3952822827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d80042c0 con 0x7f35e807ddd0 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.532+0000 7f35e5d9b700 1 -- 172.21.15.7:0/3952822827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35d8011a80 con 0x7f35e807ddd0 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.533+0000 7f35ed131700 1 -- 172.21.15.7:0/3952822827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 msgr2=0x7f35e807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.533+0000 7f35ed131700 1 --2- 172.21.15.7:0/3952822827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e807c2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f35d80098a0 tx=0x7f35d8009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.533+0000 7f35ed131700 1 -- 172.21.15.7:0/3952822827 shutdown_connections 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.533+0000 7f35ed131700 1 --2- 172.21.15.7:0/3952822827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e807c2d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.533+0000 7f35ed131700 1 -- 172.21.15.7:0/3952822827 >> 172.21.15.7:0/3952822827 conn(0x7f35e8103200 msgr2=0x7f35e8105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.533+0000 7f35ed131700 1 -- 172.21.15.7:0/3952822827 shutdown_connections 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.533+0000 7f35ed131700 1 -- 172.21.15.7:0/3952822827 wait complete. 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.533+0000 7f35ed131700 1 Processor -- start 2024-03-23T13:10:46.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.534+0000 7f35ed131700 1 -- start start 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.534+0000 7f35ed131700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.534+0000 7f35ed131700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35e819dfb0 con 0x7f35e807ddd0 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.534+0000 7f35e6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.534+0000 7f35e6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e819da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36528/0 (socket says 172.21.15.7:36528) 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.534+0000 7f35e6d9d700 1 -- 172.21.15.7:0/3142968051 learned_addr learned my addr 172.21.15.7:0/3142968051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.534+0000 7f35e6d9d700 1 -- 172.21.15.7:0/3142968051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d8009550 con 0x7f35e807ddd0 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.534+0000 7f35e6d9d700 1 --2- 172.21.15.7:0/3142968051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e819da70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35d8011740 tx=0x7f35d8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.535+0000 7f35dffff700 1 -- 172.21.15.7:0/3142968051 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35d8003b20 con 0x7f35e807ddd0 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.535+0000 7f35dffff700 1 -- 172.21.15.7:0/3142968051 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d801a910 con 0x7f35e807ddd0 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.535+0000 7f35ed131700 1 -- 172.21.15.7:0/3142968051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e819e1b0 con 0x7f35e807ddd0 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.535+0000 7f35ed131700 1 -- 172.21.15.7:0/3142968051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e819e650 con 0x7f35e807ddd0 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.535+0000 7f35dffff700 1 -- 172.21.15.7:0/3142968051 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35d8024900 con 0x7f35e807ddd0 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.536+0000 7f35dffff700 1 -- 172.21.15.7:0/3142968051 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 45941+0+0 (secure 0 0 0) 0x7f35d8024b90 con 0x7f35e807ddd0 2024-03-23T13:10:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.536+0000 7f35dffff700 1 --2- 172.21.15.7:0/3142968051 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f35d4038ee0 0x7f35d403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.536+0000 7f35ed131700 1 -- 172.21.15.7:0/3142968051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c8005360 con 0x7f35e807ddd0 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.536+0000 7f35dffff700 1 -- 172.21.15.7:0/3142968051 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f35d804d4d0 con 0x7f35e807ddd0 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.536+0000 7f35e659c700 1 --2- 172.21.15.7:0/3142968051 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f35d4038ee0 0x7f35d403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.536+0000 7f35e659c700 1 --2- 172.21.15.7:0/3142968051 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f35d4038ee0 0x7f35d403b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35d0006fd0 tx=0x7f35d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.542+0000 7f35dffff700 1 -- 172.21.15.7:0/3142968051 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f35d802d3f0 con 0x7f35e807ddd0 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:45.785+0000 7f35ed131700 1 -- 172.21.15.7:0/3142968051 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f35c8002430 con 0x7f35d4038ee0 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.031+0000 7f35dffff700 1 -- 172.21.15.7:0/3142968051 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f35c8002430 con 0x7f35d4038ee0 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.033+0000 7f35ed131700 1 -- 172.21.15.7:0/3142968051 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f35d4038ee0 msgr2=0x7f35d403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.033+0000 7f35ed131700 1 --2- 172.21.15.7:0/3142968051 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f35d4038ee0 0x7f35d403b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35d0006fd0 tx=0x7f35d0006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.033+0000 7f35ed131700 1 -- 172.21.15.7:0/3142968051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 msgr2=0x7f35e819da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.033+0000 7f35ed131700 1 --2- 172.21.15.7:0/3142968051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e819da70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35d8011740 tx=0x7f35d8011a50 comp rx=0 tx=0).stop 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.033+0000 7f35ed131700 1 -- 172.21.15.7:0/3142968051 shutdown_connections 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.033+0000 7f35ed131700 1 --2- 172.21.15.7:0/3142968051 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f35d4038ee0 0x7f35d403b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.033+0000 7f35ed131700 1 --2- 172.21.15.7:0/3142968051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e807ddd0 0x7f35e819da70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:46.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.033+0000 7f35ed131700 1 -- 172.21.15.7:0/3142968051 >> 172.21.15.7:0/3142968051 conn(0x7f35e8103200 msgr2=0x7f35e8103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:46.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.033+0000 7f35ed131700 1 -- 172.21.15.7:0/3142968051 shutdown_connections 2024-03-23T13:10:46.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.033+0000 7f35ed131700 1 -- 172.21.15.7:0/3142968051 wait complete. 2024-03-23T13:10:46.571 INFO:teuthology.orchestra.run.smithi007.stdout:Fetching dashboard port number... 2024-03-23T13:10:46.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:46 smithi007 ceph-mon[32184]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:10:46.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:47.759 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:47 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1351786000' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-23T13:10:48.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 8443 2024-03-23T13:10:48.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.996+0000 7f0461591700 1 Processor -- start 2024-03-23T13:10:48.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.996+0000 7f0461591700 1 -- start start 2024-03-23T13:10:48.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.996+0000 7f0461591700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c10aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:48.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.996+0000 7f0461591700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f045c07be40 con 0x7f045c108640 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.997+0000 7f045affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c10aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.997+0000 7f045affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c10aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36536/0 (socket says 172.21.15.7:36536) 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.997+0000 7f045affd700 1 -- 172.21.15.7:0/235323798 learned_addr learned my addr 172.21.15.7:0/235323798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.997+0000 7f045affd700 1 -- 172.21.15.7:0/235323798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c10af70 con 0x7f045c108640 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.997+0000 7f045affd700 1 --2- 172.21.15.7:0/235323798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c10aa30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f044c0098a0 tx=0x7f044c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303d3c0c476e75e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.997+0000 7f0459ffb700 1 -- 172.21.15.7:0/235323798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f044c004160 con 0x7f045c108640 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.997+0000 7f0459ffb700 1 -- 172.21.15.7:0/235323798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044c0042c0 con 0x7f045c108640 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.997+0000 7f0459ffb700 1 -- 172.21.15.7:0/235323798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f044c011a80 con 0x7f045c108640 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.998+0000 7f0461591700 1 -- 172.21.15.7:0/235323798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 msgr2=0x7f045c10aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.998+0000 7f0461591700 1 --2- 172.21.15.7:0/235323798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c10aa30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f044c0098a0 tx=0x7f044c009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.998+0000 7f0461591700 1 -- 172.21.15.7:0/235323798 shutdown_connections 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.998+0000 7f0461591700 1 --2- 172.21.15.7:0/235323798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c10aa30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.998+0000 7f0461591700 1 -- 172.21.15.7:0/235323798 >> 172.21.15.7:0/235323798 conn(0x7f045c102030 msgr2=0x7f045c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:48.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.998+0000 7f0461591700 1 -- 172.21.15.7:0/235323798 shutdown_connections 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.998+0000 7f0461591700 1 -- 172.21.15.7:0/235323798 wait complete. 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.999+0000 7f0461591700 1 Processor -- start 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.999+0000 7f0461591700 1 -- start start 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.999+0000 7f0461591700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c107ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.999+0000 7f0461591700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f045c108400 con 0x7f045c108640 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.999+0000 7f045affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c107ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.999+0000 7f045affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c107ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36544/0 (socket says 172.21.15.7:36544) 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:46.999+0000 7f045affd700 1 -- 172.21.15.7:0/1351786000 learned_addr learned my addr 172.21.15.7:0/1351786000 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.000+0000 7f045affd700 1 -- 172.21.15.7:0/1351786000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c009550 con 0x7f045c108640 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.000+0000 7f045affd700 1 --2- 172.21.15.7:0/1351786000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c107ec0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f044c011740 tx=0x7f044c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.000+0000 7f0443fff700 1 -- 172.21.15.7:0/1351786000 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f044c003b20 con 0x7f045c108640 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.000+0000 7f0443fff700 1 -- 172.21.15.7:0/1351786000 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044c01a910 con 0x7f045c108640 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.000+0000 7f0443fff700 1 -- 172.21.15.7:0/1351786000 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f044c024900 con 0x7f045c108640 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.000+0000 7f0461591700 1 -- 172.21.15.7:0/1351786000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045c1065f0 con 0x7f045c108640 2024-03-23T13:10:48.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.000+0000 7f0461591700 1 -- 172.21.15.7:0/1351786000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045c106a90 con 0x7f045c108640 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.001+0000 7f0461591700 1 -- 172.21.15.7:0/1351786000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f045c10bdb0 con 0x7f045c108640 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.001+0000 7f0443fff700 1 -- 172.21.15.7:0/1351786000 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 45941+0+0 (secure 0 0 0) 0x7f044c01aa80 con 0x7f045c108640 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.001+0000 7f0443fff700 1 --2- 172.21.15.7:0/1351786000 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0444038f40 0x7f044403b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.001+0000 7f0443fff700 1 -- 172.21.15.7:0/1351786000 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f044c04d3d0 con 0x7f045c108640 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.002+0000 7f045a7fc700 1 --2- 172.21.15.7:0/1351786000 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0444038f40 0x7f044403b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.002+0000 7f045a7fc700 1 --2- 172.21.15.7:0/1351786000 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0444038f40 0x7f044403b400 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0450006fd0 tx=0x7f0450006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.006+0000 7f0443fff700 1 -- 172.21.15.7:0/1351786000 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f044c01ae10 con 0x7f045c108640 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.213+0000 7f0461591700 1 -- 172.21.15.7:0/1351786000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f045c10bfc0 con 0x7f045c108640 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.214+0000 7f0443fff700 1 -- 172.21.15.7:0/1351786000 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f044c018350 con 0x7f045c108640 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.216+0000 7f0461591700 1 -- 172.21.15.7:0/1351786000 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0444038f40 msgr2=0x7f044403b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.216+0000 7f0461591700 1 --2- 172.21.15.7:0/1351786000 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0444038f40 0x7f044403b400 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0450006fd0 tx=0x7f0450006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.216+0000 7f0461591700 1 -- 172.21.15.7:0/1351786000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 msgr2=0x7f045c107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.216+0000 7f0461591700 1 --2- 172.21.15.7:0/1351786000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c107ec0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f044c011740 tx=0x7f044c011a50 comp rx=0 tx=0).stop 2024-03-23T13:10:48.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.216+0000 7f0461591700 1 -- 172.21.15.7:0/1351786000 shutdown_connections 2024-03-23T13:10:48.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.216+0000 7f0461591700 1 --2- 172.21.15.7:0/1351786000 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0444038f40 0x7f044403b400 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:48.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.216+0000 7f0461591700 1 --2- 172.21.15.7:0/1351786000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f045c108640 0x7f045c107ec0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:48.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.216+0000 7f0461591700 1 -- 172.21.15.7:0/1351786000 >> 172.21.15.7:0/1351786000 conn(0x7f045c102030 msgr2=0x7f045c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:48.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.216+0000 7f0461591700 1 -- 172.21.15.7:0/1351786000 shutdown_connections 2024-03-23T13:10:48.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:47.216+0000 7f0461591700 1 -- 172.21.15.7:0/1351786000 wait complete. 2024-03-23T13:10:48.018 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T13:10:48.019 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-23T13:10:48.026 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T13:10:48.027 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-23T13:10:48.027 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-23T13:10:48.027 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-23T13:10:48.029 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph Dashboard is now available at: 2024-03-23T13:10:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:10:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: URL: https://smithi007.front.sepia.ceph.com:8443/ 2024-03-23T13:10:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: User: admin 2024-03-23T13:10:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: Password: lmyykqlmpz 2024-03-23T13:10:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:10:48.030 INFO:teuthology.orchestra.run.smithi007.stdout:Saving cluster configuration to /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config directory 2024-03-23T13:10:48.030 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling autotune for osd_memory_target 2024-03-23T13:10:49.266 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.481+0000 7fee37fa1700 1 Processor -- start 2024-03-23T13:10:49.266 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.482+0000 7fee37fa1700 1 -- start start 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.482+0000 7fee37fa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee30108420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.482+0000 7fee37fa1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee301089f0 con 0x7fee30108000 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.482+0000 7fee35d3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee30108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.482+0000 7fee35d3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee30108420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36556/0 (socket says 172.21.15.7:36556) 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.482+0000 7fee35d3d700 1 -- 172.21.15.7:0/3200378169 learned_addr learned my addr 172.21.15.7:0/3200378169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.482+0000 7fee35d3d700 1 -- 172.21.15.7:0/3200378169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee30106c60 con 0x7fee30108000 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee35d3d700 1 --2- 172.21.15.7:0/3200378169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee30108420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00bb40 tx=0x7fee2c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbe366e8eda0d71 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee34d3b700 1 -- 172.21.15.7:0/3200378169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee2c00dbe0 con 0x7fee30108000 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee34d3b700 1 -- 172.21.15.7:0/3200378169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee2c004510 con 0x7fee30108000 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee34d3b700 1 -- 172.21.15.7:0/3200378169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee2c010470 con 0x7fee30108000 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee37fa1700 1 -- 172.21.15.7:0/3200378169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 msgr2=0x7fee30108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee37fa1700 1 --2- 172.21.15.7:0/3200378169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee30108420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00bb40 tx=0x7fee2c00be50 comp rx=0 tx=0).stop 2024-03-23T13:10:49.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee37fa1700 1 -- 172.21.15.7:0/3200378169 shutdown_connections 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee37fa1700 1 --2- 172.21.15.7:0/3200378169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee30108420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee37fa1700 1 -- 172.21.15.7:0/3200378169 >> 172.21.15.7:0/3200378169 conn(0x7fee301017d0 msgr2=0x7fee30103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee37fa1700 1 -- 172.21.15.7:0/3200378169 shutdown_connections 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.483+0000 7fee37fa1700 1 -- 172.21.15.7:0/3200378169 wait complete. 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.484+0000 7fee37fa1700 1 Processor -- start 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.484+0000 7fee37fa1700 1 -- start start 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.484+0000 7fee37fa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee301996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.484+0000 7fee37fa1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee30199c20 con 0x7fee30108000 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.484+0000 7fee35d3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee301996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.484+0000 7fee35d3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee301996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36562/0 (socket says 172.21.15.7:36562) 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.484+0000 7fee35d3d700 1 -- 172.21.15.7:0/2671379993 learned_addr learned my addr 172.21.15.7:0/2671379993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.484+0000 7fee35d3d700 1 -- 172.21.15.7:0/2671379993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c00b7f0 con 0x7fee30108000 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.485+0000 7fee35d3d700 1 --2- 172.21.15.7:0/2671379993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee301996e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00b340 tx=0x7fee2c004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.485+0000 7fee26ffd700 1 -- 172.21.15.7:0/2671379993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee2c01e070 con 0x7fee30108000 2024-03-23T13:10:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.485+0000 7fee26ffd700 1 -- 172.21.15.7:0/2671379993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee2c010d40 con 0x7fee30108000 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.485+0000 7fee37fa1700 1 -- 172.21.15.7:0/2671379993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee30199e20 con 0x7fee30108000 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.485+0000 7fee26ffd700 1 -- 172.21.15.7:0/2671379993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee2c022b30 con 0x7fee30108000 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.485+0000 7fee37fa1700 1 -- 172.21.15.7:0/2671379993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3019a2c0 con 0x7fee30108000 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.486+0000 7fee26ffd700 1 -- 172.21.15.7:0/2671379993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7fee2c010860 con 0x7fee30108000 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.486+0000 7fee37fa1700 1 -- 172.21.15.7:0/2671379993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee30105b50 con 0x7fee30108000 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.486+0000 7fee26ffd700 1 --2- 172.21.15.7:0/2671379993 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fee1c038fe0 0x7fee1c03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.486+0000 7fee26ffd700 1 -- 172.21.15.7:0/2671379993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fee2c04fa80 con 0x7fee30108000 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.486+0000 7fee3553c700 1 --2- 172.21.15.7:0/2671379993 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fee1c038fe0 0x7fee1c03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.486+0000 7fee3553c700 1 --2- 172.21.15.7:0/2671379993 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fee1c038fe0 0x7fee1c03b4a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee20006fd0 tx=0x7fee20006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.491+0000 7fee26ffd700 1 -- 172.21.15.7:0/2671379993 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fee2c02a360 con 0x7fee30108000 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.693+0000 7fee37fa1700 1 -- 172.21.15.7:0/2671379993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fee30192db0 con 0x7fee30108000 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.693+0000 7fee26ffd700 1 -- 172.21.15.7:0/2671379993 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fee2c027020 con 0x7fee30108000 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.695+0000 7fee37fa1700 1 -- 172.21.15.7:0/2671379993 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fee1c038fe0 msgr2=0x7fee1c03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.695+0000 7fee37fa1700 1 --2- 172.21.15.7:0/2671379993 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fee1c038fe0 0x7fee1c03b4a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee20006fd0 tx=0x7fee20006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.695+0000 7fee37fa1700 1 -- 172.21.15.7:0/2671379993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 msgr2=0x7fee301996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.695+0000 7fee37fa1700 1 --2- 172.21.15.7:0/2671379993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee301996e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00b340 tx=0x7fee2c004020 comp rx=0 tx=0).stop 2024-03-23T13:10:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.695+0000 7fee37fa1700 1 -- 172.21.15.7:0/2671379993 shutdown_connections 2024-03-23T13:10:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.695+0000 7fee37fa1700 1 --2- 172.21.15.7:0/2671379993 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fee1c038fe0 0x7fee1c03b4a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.695+0000 7fee37fa1700 1 --2- 172.21.15.7:0/2671379993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee30108000 0x7fee301996e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.695+0000 7fee37fa1700 1 -- 172.21.15.7:0/2671379993 >> 172.21.15.7:0/2671379993 conn(0x7fee301017d0 msgr2=0x7fee3018fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.695+0000 7fee37fa1700 1 -- 172.21.15.7:0/2671379993 shutdown_connections 2024-03-23T13:10:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:48.695+0000 7fee37fa1700 1 -- 172.21.15.7:0/2671379993 wait complete. 2024-03-23T13:10:49.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:49 smithi007 ceph-mon[32184]: mgrmap e13: x(active, since 6s) 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.710+0000 7f88aab4d700 1 Processor -- start 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.710+0000 7f88aab4d700 1 -- start start 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.710+0000 7f88aab4d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a4108000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.710+0000 7f88aab4d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41085d0 con 0x7f88a4107be0 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.710+0000 7f88a88e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a4108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.710+0000 7f88a88e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a4108000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36570/0 (socket says 172.21.15.7:36570) 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.710+0000 7f88a88e9700 1 -- 172.21.15.7:0/2368697368 learned_addr learned my addr 172.21.15.7:0/2368697368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.710+0000 7f88a88e9700 1 -- 172.21.15.7:0/2368697368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a4108d40 con 0x7f88a4107be0 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.710+0000 7f88a88e9700 1 --2- 172.21.15.7:0/2368697368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a4108000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88980098a0 tx=0x7f8898009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d8e15323feeef3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.711+0000 7f88a37fe700 1 -- 172.21.15.7:0/2368697368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8898004160 con 0x7f88a4107be0 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.711+0000 7f88a37fe700 1 -- 172.21.15.7:0/2368697368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88980042c0 con 0x7f88a4107be0 2024-03-23T13:10:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.711+0000 7f88a37fe700 1 -- 172.21.15.7:0/2368697368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8898011a80 con 0x7f88a4107be0 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.711+0000 7f88aab4d700 1 -- 172.21.15.7:0/2368697368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 msgr2=0x7f88a4108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.711+0000 7f88aab4d700 1 --2- 172.21.15.7:0/2368697368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a4108000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88980098a0 tx=0x7f8898009bb0 comp rx=0 tx=0).stop 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.711+0000 7f88aab4d700 1 -- 172.21.15.7:0/2368697368 shutdown_connections 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.711+0000 7f88aab4d700 1 --2- 172.21.15.7:0/2368697368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a4108000 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.711+0000 7f88aab4d700 1 -- 172.21.15.7:0/2368697368 >> 172.21.15.7:0/2368697368 conn(0x7f88a4103200 msgr2=0x7f88a4105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.711+0000 7f88aab4d700 1 -- 172.21.15.7:0/2368697368 shutdown_connections 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.711+0000 7f88aab4d700 1 -- 172.21.15.7:0/2368697368 wait complete. 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.712+0000 7f88aab4d700 1 Processor -- start 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.712+0000 7f88aab4d700 1 -- start start 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.712+0000 7f88aab4d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.712+0000 7f88aab4d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88a419be10 con 0x7f88a4107be0 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.712+0000 7f88a88e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.712+0000 7f88a88e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a419b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36586/0 (socket says 172.21.15.7:36586) 2024-03-23T13:10:50.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.712+0000 7f88a88e9700 1 -- 172.21.15.7:0/2760516584 learned_addr learned my addr 172.21.15.7:0/2760516584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.712+0000 7f88a88e9700 1 -- 172.21.15.7:0/2760516584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8898009550 con 0x7f88a4107be0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.712+0000 7f88a88e9700 1 --2- 172.21.15.7:0/2760516584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a419b8d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8898011740 tx=0x7f8898011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.713+0000 7f88a1ffb700 1 -- 172.21.15.7:0/2760516584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88980039e0 con 0x7f88a4107be0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.713+0000 7f88aab4d700 1 -- 172.21.15.7:0/2760516584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a419c010 con 0x7f88a4107be0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.713+0000 7f88a1ffb700 1 -- 172.21.15.7:0/2760516584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8898003b40 con 0x7f88a4107be0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.713+0000 7f88a1ffb700 1 -- 172.21.15.7:0/2760516584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8898024710 con 0x7f88a4107be0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.713+0000 7f88aab4d700 1 -- 172.21.15.7:0/2760516584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a419c4b0 con 0x7f88a4107be0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.714+0000 7f88a1ffb700 1 -- 172.21.15.7:0/2760516584 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7f889801ab30 con 0x7f88a4107be0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.714+0000 7f88aab4d700 1 -- 172.21.15.7:0/2760516584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a410bf70 con 0x7f88a4107be0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.714+0000 7f88a1ffb700 1 --2- 172.21.15.7:0/2760516584 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8894038f80 0x7f889403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.714+0000 7f88a1ffb700 1 -- 172.21.15.7:0/2760516584 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f889804d3d0 con 0x7f88a4107be0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.714+0000 7f88a3fff700 1 --2- 172.21.15.7:0/2760516584 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8894038f80 0x7f889403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.714+0000 7f88a3fff700 1 --2- 172.21.15.7:0/2760516584 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8894038f80 0x7f889403b440 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8890006fd0 tx=0x7f8890006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:50.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:49.719+0000 7f88a1ffb700 1 -- 172.21.15.7:0/2760516584 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f889801a950 con 0x7f88a4107be0 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.040+0000 7f88aab4d700 1 -- 172.21.15.7:0/2760516584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f88a419ed90 con 0x7f88a4107be0 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.043+0000 7f88a1ffb700 1 -- 172.21.15.7:0/2760516584 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f889804cc60 con 0x7f88a4107be0 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.045+0000 7f88aab4d700 1 -- 172.21.15.7:0/2760516584 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8894038f80 msgr2=0x7f889403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.045+0000 7f88aab4d700 1 --2- 172.21.15.7:0/2760516584 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8894038f80 0x7f889403b440 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8890006fd0 tx=0x7f8890006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.045+0000 7f88aab4d700 1 -- 172.21.15.7:0/2760516584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 msgr2=0x7f88a419b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.045+0000 7f88aab4d700 1 --2- 172.21.15.7:0/2760516584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a419b8d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8898011740 tx=0x7f8898011b00 comp rx=0 tx=0).stop 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.045+0000 7f88aab4d700 1 -- 172.21.15.7:0/2760516584 shutdown_connections 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.045+0000 7f88aab4d700 1 --2- 172.21.15.7:0/2760516584 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8894038f80 0x7f889403b440 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.045+0000 7f88aab4d700 1 --2- 172.21.15.7:0/2760516584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88a4107be0 0x7f88a419b8d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.045+0000 7f88aab4d700 1 -- 172.21.15.7:0/2760516584 >> 172.21.15.7:0/2760516584 conn(0x7f88a4103200 msgr2=0x7f88a4103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.045+0000 7f88aab4d700 1 -- 172.21.15.7:0/2760516584 shutdown_connections 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-23T13:10:50.045+0000 7f88aab4d700 1 -- 172.21.15.7:0/2760516584 wait complete. 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-23T13:10:50.777 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout:Or, if you are only running a single cluster on this host: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: ceph telemetry on 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout:For more information see: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:10:50.778 INFO:teuthology.orchestra.run.smithi007.stdout:Bootstrap complete. 2024-03-23T13:10:50.819 INFO:tasks.cephadm:Fetching config... 2024-03-23T13:10:50.819 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:10:50.819 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-23T13:10:50.835 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-23T13:10:50.835 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:10:50.835 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-23T13:10:50.891 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-23T13:10:50.891 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:10:50.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-23T13:10:50.959 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-23T13:10:50.959 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:10:50.959 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-23T13:10:51.014 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-23T13:10:51.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxg0h2LByN/B4Tw3qoUrmdrESxVNkRctLZadEsBY2ltlJP8fFMowGqm6UoWaOJoZd4CQYYYWpmlr58tQM1J/MJGSKHV1AknLIMz3K8YD3vR/4SEZoi7bEawMkxlQPGIxhT+e5jIhvHwmlEcch+F7LCMxuaB1Le5hcrQhAUIeYEv7oJqPM31FvYBvQJeDpePZodv2DAgdzyL0w8XoOORnEd2VGK1iJzgf0e2f+EWQoa0ihCmqvjSC+KBhGnt8wzdFQKkYI3YqUMrCLh4LJ3bZQDE5y1+o8bgih753PC2Zzckmnff8U1N3V8dxMjiMajOOqwyYsfaImcEqUqsVXGfEDfdyFe14VcwuHcfB2yJ1Sq6JHesM8Utz5ZDHOhajDj648gAEYLOxdh7EptRONGoT5Tet+JHuYkiEJi4OxAmg6A0lEu9pI5J/xfJXIPpYXqowpe7uOs+toWKMOFMO0haIKQD6OJY3gSs8ZDX800PL4XM170V/PwigVoUP9c+2tmoM8= ceph-745f591a-e916-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T13:10:51.094 INFO:teuthology.orchestra.run.smithi007.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxg0h2LByN/B4Tw3qoUrmdrESxVNkRctLZadEsBY2ltlJP8fFMowGqm6UoWaOJoZd4CQYYYWpmlr58tQM1J/MJGSKHV1AknLIMz3K8YD3vR/4SEZoi7bEawMkxlQPGIxhT+e5jIhvHwmlEcch+F7LCMxuaB1Le5hcrQhAUIeYEv7oJqPM31FvYBvQJeDpePZodv2DAgdzyL0w8XoOORnEd2VGK1iJzgf0e2f+EWQoa0ihCmqvjSC+KBhGnt8wzdFQKkYI3YqUMrCLh4LJ3bZQDE5y1+o8bgih753PC2Zzckmnff8U1N3V8dxMjiMajOOqwyYsfaImcEqUqsVXGfEDfdyFe14VcwuHcfB2yJ1Sq6JHesM8Utz5ZDHOhajDj648gAEYLOxdh7EptRONGoT5Tet+JHuYkiEJi4OxAmg6A0lEu9pI5J/xfJXIPpYXqowpe7uOs+toWKMOFMO0haIKQD6OJY3gSs8ZDX800PL4XM170V/PwigVoUP9c+2tmoM8= ceph-745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:10:51.112 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxg0h2LByN/B4Tw3qoUrmdrESxVNkRctLZadEsBY2ltlJP8fFMowGqm6UoWaOJoZd4CQYYYWpmlr58tQM1J/MJGSKHV1AknLIMz3K8YD3vR/4SEZoi7bEawMkxlQPGIxhT+e5jIhvHwmlEcch+F7LCMxuaB1Le5hcrQhAUIeYEv7oJqPM31FvYBvQJeDpePZodv2DAgdzyL0w8XoOORnEd2VGK1iJzgf0e2f+EWQoa0ihCmqvjSC+KBhGnt8wzdFQKkYI3YqUMrCLh4LJ3bZQDE5y1+o8bgih753PC2Zzckmnff8U1N3V8dxMjiMajOOqwyYsfaImcEqUqsVXGfEDfdyFe14VcwuHcfB2yJ1Sq6JHesM8Utz5ZDHOhajDj648gAEYLOxdh7EptRONGoT5Tet+JHuYkiEJi4OxAmg6A0lEu9pI5J/xfJXIPpYXqowpe7uOs+toWKMOFMO0haIKQD6OJY3gSs8ZDX800PL4XM170V/PwigVoUP9c+2tmoM8= ceph-745f591a-e916-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T13:10:51.159 INFO:teuthology.orchestra.run.smithi130.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxg0h2LByN/B4Tw3qoUrmdrESxVNkRctLZadEsBY2ltlJP8fFMowGqm6UoWaOJoZd4CQYYYWpmlr58tQM1J/MJGSKHV1AknLIMz3K8YD3vR/4SEZoi7bEawMkxlQPGIxhT+e5jIhvHwmlEcch+F7LCMxuaB1Le5hcrQhAUIeYEv7oJqPM31FvYBvQJeDpePZodv2DAgdzyL0w8XoOORnEd2VGK1iJzgf0e2f+EWQoa0ihCmqvjSC+KBhGnt8wzdFQKkYI3YqUMrCLh4LJ3bZQDE5y1+o8bgih753PC2Zzckmnff8U1N3V8dxMjiMajOOqwyYsfaImcEqUqsVXGfEDfdyFe14VcwuHcfB2yJ1Sq6JHesM8Utz5ZDHOhajDj648gAEYLOxdh7EptRONGoT5Tet+JHuYkiEJi4OxAmg6A0lEu9pI5J/xfJXIPpYXqowpe7uOs+toWKMOFMO0haIKQD6OJY3gSs8ZDX800PL4XM170V/PwigVoUP9c+2tmoM8= ceph-745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:10:51.173 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxg0h2LByN/B4Tw3qoUrmdrESxVNkRctLZadEsBY2ltlJP8fFMowGqm6UoWaOJoZd4CQYYYWpmlr58tQM1J/MJGSKHV1AknLIMz3K8YD3vR/4SEZoi7bEawMkxlQPGIxhT+e5jIhvHwmlEcch+F7LCMxuaB1Le5hcrQhAUIeYEv7oJqPM31FvYBvQJeDpePZodv2DAgdzyL0w8XoOORnEd2VGK1iJzgf0e2f+EWQoa0ihCmqvjSC+KBhGnt8wzdFQKkYI3YqUMrCLh4LJ3bZQDE5y1+o8bgih753PC2Zzckmnff8U1N3V8dxMjiMajOOqwyYsfaImcEqUqsVXGfEDfdyFe14VcwuHcfB2yJ1Sq6JHesM8Utz5ZDHOhajDj648gAEYLOxdh7EptRONGoT5Tet+JHuYkiEJi4OxAmg6A0lEu9pI5J/xfJXIPpYXqowpe7uOs+toWKMOFMO0haIKQD6OJY3gSs8ZDX800PL4XM170V/PwigVoUP9c+2tmoM8= ceph-745f591a-e916-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T13:10:51.221 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxg0h2LByN/B4Tw3qoUrmdrESxVNkRctLZadEsBY2ltlJP8fFMowGqm6UoWaOJoZd4CQYYYWpmlr58tQM1J/MJGSKHV1AknLIMz3K8YD3vR/4SEZoi7bEawMkxlQPGIxhT+e5jIhvHwmlEcch+F7LCMxuaB1Le5hcrQhAUIeYEv7oJqPM31FvYBvQJeDpePZodv2DAgdzyL0w8XoOORnEd2VGK1iJzgf0e2f+EWQoa0ihCmqvjSC+KBhGnt8wzdFQKkYI3YqUMrCLh4LJ3bZQDE5y1+o8bgih753PC2Zzckmnff8U1N3V8dxMjiMajOOqwyYsfaImcEqUqsVXGfEDfdyFe14VcwuHcfB2yJ1Sq6JHesM8Utz5ZDHOhajDj648gAEYLOxdh7EptRONGoT5Tet+JHuYkiEJi4OxAmg6A0lEu9pI5J/xfJXIPpYXqowpe7uOs+toWKMOFMO0haIKQD6OJY3gSs8ZDX800PL4XM170V/PwigVoUP9c+2tmoM8= ceph-745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:10:51.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-23T13:10:51.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:51 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2760516584' entity='client.admin' 2024-03-23T13:10:51.476 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:10:53.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 -- 172.21.15.7:0/249050489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1888100520 msgr2=0x7f1888100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:53.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 --2- 172.21.15.7:0/249050489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1888100520 0x7f1888100940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1884009910 tx=0x7f1884009c20 comp rx=0 tx=0).stop 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 -- 172.21.15.7:0/249050489 shutdown_connections 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 --2- 172.21.15.7:0/249050489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1888100520 0x7f1888100940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 -- 172.21.15.7:0/249050489 >> 172.21.15.7:0/249050489 conn(0x7f18880fc100 msgr2=0x7f18880fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 -- 172.21.15.7:0/249050489 shutdown_connections 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 -- 172.21.15.7:0/249050489 wait complete. 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 Processor -- start 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 -- start start 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1888100520 0x7f1888199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.707+0000 7f188ffd2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1888199890 con 0x7f1888100520 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.708+0000 7f188dd6e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1888100520 0x7f1888199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.708+0000 7f188dd6e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1888100520 0x7f1888199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36604/0 (socket says 172.21.15.7:36604) 2024-03-23T13:10:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.708+0000 7f188dd6e700 1 -- 172.21.15.7:0/2589111445 learned_addr learned my addr 172.21.15.7:0/2589111445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.708+0000 7f188dd6e700 1 -- 172.21.15.7:0/2589111445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18840095f0 con 0x7f1888100520 2024-03-23T13:10:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.708+0000 7f188dd6e700 1 --2- 172.21.15.7:0/2589111445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1888100520 0x7f1888199350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1884004f40 tx=0x7f1884005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.708+0000 7f187effd700 1 -- 172.21.15.7:0/2589111445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f188401d070 con 0x7f1888100520 2024-03-23T13:10:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.708+0000 7f187effd700 1 -- 172.21.15.7:0/2589111445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f188400be40 con 0x7f1888100520 2024-03-23T13:10:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.708+0000 7f188ffd2700 1 -- 172.21.15.7:0/2589111445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1888199a90 con 0x7f1888100520 2024-03-23T13:10:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.708+0000 7f187effd700 1 -- 172.21.15.7:0/2589111445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f188400f460 con 0x7f1888100520 2024-03-23T13:10:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.708+0000 7f188ffd2700 1 -- 172.21.15.7:0/2589111445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1888199f30 con 0x7f1888100520 2024-03-23T13:10:53.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.709+0000 7f187effd700 1 -- 172.21.15.7:0/2589111445 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7f188400b960 con 0x7f1888100520 2024-03-23T13:10:53.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.709+0000 7f188ffd2700 1 -- 172.21.15.7:0/2589111445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1888192c90 con 0x7f1888100520 2024-03-23T13:10:53.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.709+0000 7f187effd700 1 --2- 172.21.15.7:0/2589111445 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1874038f80 0x7f187403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:53.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.709+0000 7f187effd700 1 -- 172.21.15.7:0/2589111445 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f188404d310 con 0x7f1888100520 2024-03-23T13:10:53.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.709+0000 7f188d56d700 1 --2- 172.21.15.7:0/2589111445 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1874038f80 0x7f187403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:53.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.710+0000 7f188d56d700 1 --2- 172.21.15.7:0/2589111445 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1874038f80 0x7f187403b440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1878006fd0 tx=0x7f1878006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:53.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.714+0000 7f187effd700 1 -- 172.21.15.7:0/2589111445 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f188402baa0 con 0x7f1888100520 2024-03-23T13:10:53.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.921+0000 7f188ffd2700 1 -- 172.21.15.7:0/2589111445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f1888063830 con 0x7f1888100520 2024-03-23T13:10:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.923+0000 7f187effd700 1 -- 172.21.15.7:0/2589111445 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f1884027030 con 0x7f1888100520 2024-03-23T13:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.925+0000 7f188ffd2700 1 -- 172.21.15.7:0/2589111445 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1874038f80 msgr2=0x7f187403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.925+0000 7f188ffd2700 1 --2- 172.21.15.7:0/2589111445 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1874038f80 0x7f187403b440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1878006fd0 tx=0x7f1878006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.925+0000 7f188ffd2700 1 -- 172.21.15.7:0/2589111445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1888100520 msgr2=0x7f1888199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.925+0000 7f188ffd2700 1 --2- 172.21.15.7:0/2589111445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1888100520 0x7f1888199350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1884004f40 tx=0x7f1884005020 comp rx=0 tx=0).stop 2024-03-23T13:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.925+0000 7f188ffd2700 1 -- 172.21.15.7:0/2589111445 shutdown_connections 2024-03-23T13:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.926+0000 7f188ffd2700 1 --2- 172.21.15.7:0/2589111445 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1874038f80 0x7f187403b440 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.926+0000 7f188ffd2700 1 --2- 172.21.15.7:0/2589111445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1888100520 0x7f1888199350 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.926+0000 7f188ffd2700 1 -- 172.21.15.7:0/2589111445 >> 172.21.15.7:0/2589111445 conn(0x7f18880fc100 msgr2=0x7f18880fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.926+0000 7f188ffd2700 1 -- 172.21.15.7:0/2589111445 shutdown_connections 2024-03-23T13:10:53.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:53.926+0000 7f188ffd2700 1 -- 172.21.15.7:0/2589111445 wait complete. 2024-03-23T13:10:54.718 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-23T13:10:54.718 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-23T13:10:54.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:54 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:54.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:54 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:54.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:54 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:10:54.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:54 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:54.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:54 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:54.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:54 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:54.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:54 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2589111445' entity='client.admin' 2024-03-23T13:10:54.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:54 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:54.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:54 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:54.963 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:10:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.101+0000 7f377c3f2700 1 -- 172.21.15.7:0/3677401217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3774101c50 msgr2=0x7f3774102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.101+0000 7f377c3f2700 1 --2- 172.21.15.7:0/3677401217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3774101c50 0x7f3774102070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f376c00b0a0 tx=0x7f376c00b3b0 comp rx=0 tx=0).stop 2024-03-23T13:10:57.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.101+0000 7f377c3f2700 1 -- 172.21.15.7:0/3677401217 shutdown_connections 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.101+0000 7f377c3f2700 1 --2- 172.21.15.7:0/3677401217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3774101c50 0x7f3774102070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.101+0000 7f377c3f2700 1 -- 172.21.15.7:0/3677401217 >> 172.21.15.7:0/3677401217 conn(0x7f37740fd1f0 msgr2=0x7f37740ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.101+0000 7f377c3f2700 1 -- 172.21.15.7:0/3677401217 shutdown_connections 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.101+0000 7f377c3f2700 1 -- 172.21.15.7:0/3677401217 wait complete. 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.102+0000 7f377c3f2700 1 Processor -- start 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.102+0000 7f377c3f2700 1 -- start start 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.102+0000 7f377c3f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3774101c50 0x7f3774197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.102+0000 7f377c3f2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3774197680 con 0x7f3774101c50 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.102+0000 7f377a18e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3774101c50 0x7f3774197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.102+0000 7f377a18e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3774101c50 0x7f3774197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41210/0 (socket says 172.21.15.7:41210) 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.102+0000 7f377a18e700 1 -- 172.21.15.7:0/3678560513 learned_addr learned my addr 172.21.15.7:0/3678560513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:57.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.102+0000 7f377a18e700 1 -- 172.21.15.7:0/3678560513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c009d00 con 0x7f3774101c50 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.102+0000 7f377a18e700 1 --2- 172.21.15.7:0/3678560513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3774101c50 0x7f3774197140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f376c015040 tx=0x7f376c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.102+0000 7f37737fe700 1 -- 172.21.15.7:0/3678560513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f376c00e040 con 0x7f3774101c50 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.103+0000 7f37737fe700 1 -- 172.21.15.7:0/3678560513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f376c003d40 con 0x7f3774101c50 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.103+0000 7f377c3f2700 1 -- 172.21.15.7:0/3678560513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3774197880 con 0x7f3774101c50 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.103+0000 7f37737fe700 1 -- 172.21.15.7:0/3678560513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f376c01d770 con 0x7f3774101c50 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.103+0000 7f377c3f2700 1 -- 172.21.15.7:0/3678560513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3774197d20 con 0x7f3774101c50 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.103+0000 7f37737fe700 1 -- 172.21.15.7:0/3678560513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7f376c019070 con 0x7f3774101c50 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.103+0000 7f377c3f2700 1 -- 172.21.15.7:0/3678560513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3774190ab0 con 0x7f3774101c50 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.104+0000 7f37737fe700 1 --2- 172.21.15.7:0/3678560513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f375c038f80 0x7f375c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.104+0000 7f37737fe700 1 -- 172.21.15.7:0/3678560513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f376c04d110 con 0x7f3774101c50 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.104+0000 7f377998d700 1 --2- 172.21.15.7:0/3678560513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f375c038f80 0x7f375c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.104+0000 7f377998d700 1 --2- 172.21.15.7:0/3678560513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f375c038f80 0x7f375c03b440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3768006fd0 tx=0x7f3768006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:57.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.108+0000 7f37737fe700 1 -- 172.21.15.7:0/3678560513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f376c01c5b0 con 0x7f3774101c50 2024-03-23T13:10:57.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.317+0000 7f377c3f2700 1 -- 172.21.15.7:0/3678560513 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f377402db60 con 0x7f375c038f80 2024-03-23T13:10:57.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.321+0000 7f37737fe700 1 -- 172.21.15.7:0/3678560513 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f377402db60 con 0x7f375c038f80 2024-03-23T13:10:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.323+0000 7f377c3f2700 1 -- 172.21.15.7:0/3678560513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f375c038f80 msgr2=0x7f375c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.323+0000 7f377c3f2700 1 --2- 172.21.15.7:0/3678560513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f375c038f80 0x7f375c03b440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3768006fd0 tx=0x7f3768006e40 comp rx=0 tx=0).stop 2024-03-23T13:10:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.323+0000 7f377c3f2700 1 -- 172.21.15.7:0/3678560513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3774101c50 msgr2=0x7f3774197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.323+0000 7f377c3f2700 1 --2- 172.21.15.7:0/3678560513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3774101c50 0x7f3774197140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f376c015040 tx=0x7f376c00bd40 comp rx=0 tx=0).stop 2024-03-23T13:10:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.323+0000 7f377c3f2700 1 -- 172.21.15.7:0/3678560513 shutdown_connections 2024-03-23T13:10:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.323+0000 7f377c3f2700 1 --2- 172.21.15.7:0/3678560513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f375c038f80 0x7f375c03b440 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.323+0000 7f377c3f2700 1 --2- 172.21.15.7:0/3678560513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3774101c50 0x7f3774197140 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.323+0000 7f377c3f2700 1 -- 172.21.15.7:0/3678560513 >> 172.21.15.7:0/3678560513 conn(0x7f37740fd1f0 msgr2=0x7f37740fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.323+0000 7f377c3f2700 1 -- 172.21.15.7:0/3678560513 shutdown_connections 2024-03-23T13:10:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:57.323+0000 7f377c3f2700 1 -- 172.21.15.7:0/3678560513 wait complete. 2024-03-23T13:10:58.066 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi130 2024-03-23T13:10:58.067 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:10:58.067 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.conf 2024-03-23T13:10:58.085 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:10:58.085 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T13:10:58.143 INFO:tasks.cephadm:Adding host smithi130 to orchestrator... 2024-03-23T13:10:58.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch host add smithi130 2024-03-23T13:10:58.391 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:10:58.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:10:58.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:58.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:10:58.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:10:58.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:10:58.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-23T13:10:58.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: Updating smithi007:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:10:58.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-23T13:10:58.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:58.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:58.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:10:59.566 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:10:59 smithi007 ceph-mon[32184]: Updating smithi007:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T13:10:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.786+0000 7fa648259700 1 -- 172.21.15.7:0/66252038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa640101e50 msgr2=0x7fa640102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:10:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.786+0000 7fa648259700 1 --2- 172.21.15.7:0/66252038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa640101e50 0x7fa640102230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa634009910 tx=0x7fa634009c20 comp rx=0 tx=0).stop 2024-03-23T13:10:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.786+0000 7fa648259700 1 -- 172.21.15.7:0/66252038 shutdown_connections 2024-03-23T13:10:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.786+0000 7fa648259700 1 --2- 172.21.15.7:0/66252038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa640101e50 0x7fa640102230 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:10:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.786+0000 7fa648259700 1 -- 172.21.15.7:0/66252038 >> 172.21.15.7:0/66252038 conn(0x7fa6400fd700 msgr2=0x7fa6400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:10:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.786+0000 7fa648259700 1 -- 172.21.15.7:0/66252038 shutdown_connections 2024-03-23T13:10:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.786+0000 7fa648259700 1 -- 172.21.15.7:0/66252038 wait complete. 2024-03-23T13:10:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.787+0000 7fa648259700 1 Processor -- start 2024-03-23T13:10:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.787+0000 7fa648259700 1 -- start start 2024-03-23T13:10:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.787+0000 7fa648259700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa640101e50 0x7fa640197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.787+0000 7fa648259700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa640197a60 con 0x7fa640101e50 2024-03-23T13:10:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.787+0000 7fa645ff5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa640101e50 0x7fa640197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.787+0000 7fa645ff5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa640101e50 0x7fa640197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41222/0 (socket says 172.21.15.7:41222) 2024-03-23T13:10:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.787+0000 7fa645ff5700 1 -- 172.21.15.7:0/4215541259 learned_addr learned my addr 172.21.15.7:0/4215541259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:10:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.788+0000 7fa645ff5700 1 -- 172.21.15.7:0/4215541259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6340095f0 con 0x7fa640101e50 2024-03-23T13:10:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.788+0000 7fa645ff5700 1 --2- 172.21.15.7:0/4215541259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa640101e50 0x7fa640197520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa634004f40 tx=0x7fa634005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.788+0000 7fa632ffd700 1 -- 172.21.15.7:0/4215541259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa63401d070 con 0x7fa640101e50 2024-03-23T13:10:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.788+0000 7fa632ffd700 1 -- 172.21.15.7:0/4215541259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa63400bc30 con 0x7fa640101e50 2024-03-23T13:10:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.788+0000 7fa648259700 1 -- 172.21.15.7:0/4215541259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa640197c60 con 0x7fa640101e50 2024-03-23T13:10:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.788+0000 7fa632ffd700 1 -- 172.21.15.7:0/4215541259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa634003970 con 0x7fa640101e50 2024-03-23T13:10:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.788+0000 7fa648259700 1 -- 172.21.15.7:0/4215541259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa640198080 con 0x7fa640101e50 2024-03-23T13:10:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.789+0000 7fa632ffd700 1 -- 172.21.15.7:0/4215541259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7fa634003be0 con 0x7fa640101e50 2024-03-23T13:10:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.789+0000 7fa648259700 1 -- 172.21.15.7:0/4215541259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa640063830 con 0x7fa640101e50 2024-03-23T13:10:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.789+0000 7fa632ffd700 1 --2- 172.21.15.7:0/4215541259 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa62c038fe0 0x7fa62c03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:10:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.789+0000 7fa632ffd700 1 -- 172.21.15.7:0/4215541259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa63404cf30 con 0x7fa640101e50 2024-03-23T13:10:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.789+0000 7fa6457f4700 1 --2- 172.21.15.7:0/4215541259 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa62c038fe0 0x7fa62c03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:10:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.789+0000 7fa6457f4700 1 --2- 172.21.15.7:0/4215541259 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa62c038fe0 0x7fa62c03b4a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa63c006fd0 tx=0x7fa63c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:10:59.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:10:59.794+0000 7fa632ffd700 1 -- 172.21.15.7:0/4215541259 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa634029950 con 0x7fa640101e50 2024-03-23T13:10:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:00.003+0000 7fa648259700 1 -- 172.21.15.7:0/4215541259 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}) -- 0x7fa64010c120 con 0x7fa62c038fe0 2024-03-23T13:11:01.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:00 smithi007 ceph-mon[32184]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:01.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:00 smithi007 ceph-mon[32184]: Deploying cephadm binary to smithi130 2024-03-23T13:11:03.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:02 smithi007 ceph-mon[32184]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:04.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.377+0000 7fa632ffd700 1 -- 172.21.15.7:0/4215541259 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa64010c120 con 0x7fa62c038fe0 2024-03-23T13:11:04.369 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi130' with addr '172.21.15.130' 2024-03-23T13:11:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.380+0000 7fa648259700 1 -- 172.21.15.7:0/4215541259 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa62c038fe0 msgr2=0x7fa62c03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.380+0000 7fa648259700 1 --2- 172.21.15.7:0/4215541259 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa62c038fe0 0x7fa62c03b4a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa63c006fd0 tx=0x7fa63c006e40 comp rx=0 tx=0).stop 2024-03-23T13:11:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.380+0000 7fa648259700 1 -- 172.21.15.7:0/4215541259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa640101e50 msgr2=0x7fa640197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.380+0000 7fa648259700 1 --2- 172.21.15.7:0/4215541259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa640101e50 0x7fa640197520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa634004f40 tx=0x7fa634005020 comp rx=0 tx=0).stop 2024-03-23T13:11:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.380+0000 7fa648259700 1 -- 172.21.15.7:0/4215541259 shutdown_connections 2024-03-23T13:11:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.380+0000 7fa648259700 1 --2- 172.21.15.7:0/4215541259 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa62c038fe0 0x7fa62c03b4a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.380+0000 7fa648259700 1 --2- 172.21.15.7:0/4215541259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa640101e50 0x7fa640197520 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.380+0000 7fa648259700 1 -- 172.21.15.7:0/4215541259 >> 172.21.15.7:0/4215541259 conn(0x7fa6400fd700 msgr2=0x7fa640103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.380+0000 7fa648259700 1 -- 172.21.15.7:0/4215541259 shutdown_connections 2024-03-23T13:11:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:04.380+0000 7fa648259700 1 -- 172.21.15.7:0/4215541259 wait complete. 2024-03-23T13:11:05.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-23T13:11:05.279 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:11:05.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:05 smithi007 ceph-mon[32184]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:05.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:05.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:05 smithi007 ceph-mon[32184]: Added host smithi130 2024-03-23T13:11:05.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:05.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:06.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:06.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.849+0000 7fd96b6e3700 1 -- 172.21.15.7:0/2006182354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964101e40 msgr2=0x7fd964102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:06.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.849+0000 7fd96b6e3700 1 --2- 172.21.15.7:0/2006182354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964101e40 0x7fd964102220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00b0a0 tx=0x7fd95c00b3b0 comp rx=0 tx=0).stop 2024-03-23T13:11:06.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.849+0000 7fd96b6e3700 1 -- 172.21.15.7:0/2006182354 shutdown_connections 2024-03-23T13:11:06.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.849+0000 7fd96b6e3700 1 --2- 172.21.15.7:0/2006182354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964101e40 0x7fd964102220 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:06.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.849+0000 7fd96b6e3700 1 -- 172.21.15.7:0/2006182354 >> 172.21.15.7:0/2006182354 conn(0x7fd9640fd710 msgr2=0x7fd9640ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:06.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.849+0000 7fd96b6e3700 1 -- 172.21.15.7:0/2006182354 shutdown_connections 2024-03-23T13:11:06.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.849+0000 7fd96b6e3700 1 -- 172.21.15.7:0/2006182354 wait complete. 2024-03-23T13:11:06.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.849+0000 7fd96b6e3700 1 Processor -- start 2024-03-23T13:11:06.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd96b6e3700 1 -- start start 2024-03-23T13:11:06.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd96b6e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964101e40 0x7fd96419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:06.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd96b6e3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd96419f920 con 0x7fd964101e40 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd96947f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964101e40 0x7fd96419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd96947f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964101e40 0x7fd96419f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43726/0 (socket says 172.21.15.7:43726) 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd96947f700 1 -- 172.21.15.7:0/3097805882 learned_addr learned my addr 172.21.15.7:0/3097805882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd96947f700 1 -- 172.21.15.7:0/3097805882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c009d00 con 0x7fd964101e40 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd96947f700 1 --2- 172.21.15.7:0/3097805882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964101e40 0x7fd96419f3e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00b830 tx=0x7fd95c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd9627fc700 1 -- 172.21.15.7:0/3097805882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd95c00e040 con 0x7fd964101e40 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd96b6e3700 1 -- 172.21.15.7:0/3097805882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96419fb20 con 0x7fd964101e40 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd9627fc700 1 -- 172.21.15.7:0/3097805882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd95c003c10 con 0x7fd964101e40 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.850+0000 7fd9627fc700 1 -- 172.21.15.7:0/3097805882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd95c01d6b0 con 0x7fd964101e40 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.851+0000 7fd96b6e3700 1 -- 172.21.15.7:0/3097805882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96419ff40 con 0x7fd964101e40 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.851+0000 7fd9627fc700 1 -- 172.21.15.7:0/3097805882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7fd95c019070 con 0x7fd964101e40 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.851+0000 7fd9627fc700 1 --2- 172.21.15.7:0/3097805882 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd94c038f80 0x7fd94c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.851+0000 7fd96b6e3700 1 -- 172.21.15.7:0/3097805882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd964063830 con 0x7fd964101e40 2024-03-23T13:11:06.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.852+0000 7fd9627fc700 1 -- 172.21.15.7:0/3097805882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd95c04d0f0 con 0x7fd964101e40 2024-03-23T13:11:06.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.852+0000 7fd968c7e700 1 --2- 172.21.15.7:0/3097805882 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd94c038f80 0x7fd94c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:06.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.852+0000 7fd968c7e700 1 --2- 172.21.15.7:0/3097805882 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd94c038f80 0x7fd94c03b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd958006fd0 tx=0x7fd958006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:06.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:06.857+0000 7fd9627fc700 1 -- 172.21.15.7:0/3097805882 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd95c017030 con 0x7fd964101e40 2024-03-23T13:11:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.066+0000 7fd96b6e3700 1 -- 172.21.15.7:0/3097805882 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd96410c070 con 0x7fd94c038f80 2024-03-23T13:11:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.068+0000 7fd9627fc700 1 -- 172.21.15.7:0/3097805882 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fd96410c070 con 0x7fd94c038f80 2024-03-23T13:11:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:11:07.059 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.130", "hostname": "smithi130", "labels": [], "status": ""}] 2024-03-23T13:11:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.070+0000 7fd96b6e3700 1 -- 172.21.15.7:0/3097805882 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd94c038f80 msgr2=0x7fd94c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.070+0000 7fd96b6e3700 1 --2- 172.21.15.7:0/3097805882 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd94c038f80 0x7fd94c03b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd958006fd0 tx=0x7fd958006e40 comp rx=0 tx=0).stop 2024-03-23T13:11:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.070+0000 7fd96b6e3700 1 -- 172.21.15.7:0/3097805882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964101e40 msgr2=0x7fd96419f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.070+0000 7fd96b6e3700 1 --2- 172.21.15.7:0/3097805882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964101e40 0x7fd96419f3e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00b830 tx=0x7fd95c00bd40 comp rx=0 tx=0).stop 2024-03-23T13:11:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.070+0000 7fd96b6e3700 1 -- 172.21.15.7:0/3097805882 shutdown_connections 2024-03-23T13:11:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.070+0000 7fd96b6e3700 1 --2- 172.21.15.7:0/3097805882 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd94c038f80 0x7fd94c03b440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.070+0000 7fd96b6e3700 1 --2- 172.21.15.7:0/3097805882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964101e40 0x7fd96419f3e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.070+0000 7fd96b6e3700 1 -- 172.21.15.7:0/3097805882 >> 172.21.15.7:0/3097805882 conn(0x7fd9640fd710 msgr2=0x7fd964074430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.070+0000 7fd96b6e3700 1 -- 172.21.15.7:0/3097805882 shutdown_connections 2024-03-23T13:11:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:07.070+0000 7fd96b6e3700 1 -- 172.21.15.7:0/3097805882 wait complete. 2024-03-23T13:11:07.622 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:07 smithi007 ceph-mon[32184]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:07.622 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:07 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:07.649 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-03-23T13:11:07.649 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:11:07.650 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-03-23T13:11:07.668 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:11:07.668 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T13:11:07.725 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-03-23T13:11:07.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch host add smithi155 2024-03-23T13:11:07.964 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:11:08.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:08 smithi007 ceph-mon[32184]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:11:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.481+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3720492402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc84106ac0 msgr2=0x7fcc84106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.481+0000 7fcc8c27a700 1 --2- 172.21.15.7:0/3720492402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc84106ac0 0x7fcc84106ea0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009910 tx=0x7fcc74009c20 comp rx=0 tx=0).stop 2024-03-23T13:11:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.481+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3720492402 shutdown_connections 2024-03-23T13:11:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.481+0000 7fcc8c27a700 1 --2- 172.21.15.7:0/3720492402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc84106ac0 0x7fcc84106ea0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.481+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3720492402 >> 172.21.15.7:0/3720492402 conn(0x7fcc840fc5b0 msgr2=0x7fcc840fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.481+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3720492402 shutdown_connections 2024-03-23T13:11:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.481+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3720492402 wait complete. 2024-03-23T13:11:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.482+0000 7fcc8c27a700 1 Processor -- start 2024-03-23T13:11:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.482+0000 7fcc8c27a700 1 -- start start 2024-03-23T13:11:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.482+0000 7fcc8c27a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc84106ac0 0x7fcc840fca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.482+0000 7fcc8c27a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc840fe950 con 0x7fcc84106ac0 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.482+0000 7fcc8a016700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc84106ac0 0x7fcc840fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.482+0000 7fcc8a016700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc84106ac0 0x7fcc840fca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43732/0 (socket says 172.21.15.7:43732) 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.482+0000 7fcc8a016700 1 -- 172.21.15.7:0/3920713600 learned_addr learned my addr 172.21.15.7:0/3920713600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.482+0000 7fcc8a016700 1 -- 172.21.15.7:0/3920713600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc740095f0 con 0x7fcc84106ac0 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.483+0000 7fcc8a016700 1 --2- 172.21.15.7:0/3920713600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc84106ac0 0x7fcc840fca60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc74004f40 tx=0x7fcc74005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.483+0000 7fcc7b7fe700 1 -- 172.21.15.7:0/3920713600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc7401d070 con 0x7fcc84106ac0 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.483+0000 7fcc7b7fe700 1 -- 172.21.15.7:0/3920713600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7400bbd0 con 0x7fcc84106ac0 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.483+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3920713600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc840fcfa0 con 0x7fcc84106ac0 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.483+0000 7fcc7b7fe700 1 -- 172.21.15.7:0/3920713600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc74003900 con 0x7fcc84106ac0 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.483+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3920713600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc840fd420 con 0x7fcc84106ac0 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.484+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3920713600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc84102200 con 0x7fcc84106ac0 2024-03-23T13:11:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.484+0000 7fcc7b7fe700 1 -- 172.21.15.7:0/3920713600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7fcc7400bd40 con 0x7fcc84106ac0 2024-03-23T13:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.484+0000 7fcc7b7fe700 1 --2- 172.21.15.7:0/3920713600 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc70038fe0 0x7fcc7003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.484+0000 7fcc7b7fe700 1 -- 172.21.15.7:0/3920713600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcc7404cc90 con 0x7fcc84106ac0 2024-03-23T13:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.484+0000 7fcc89815700 1 --2- 172.21.15.7:0/3920713600 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc70038fe0 0x7fcc7003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.485+0000 7fcc89815700 1 --2- 172.21.15.7:0/3920713600 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc70038fe0 0x7fcc7003b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc80006fd0 tx=0x7fcc80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.489+0000 7fcc7b7fe700 1 -- 172.21.15.7:0/3920713600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fcc7400e5c0 con 0x7fcc84106ac0 2024-03-23T13:11:09.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:09 smithi007 ceph-mon[32184]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:09.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:09.698+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3920713600 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) -- 0x7fcc840fd6d0 con 0x7fcc70038fe0 2024-03-23T13:11:10.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:10 smithi007 ceph-mon[32184]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:10.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:10.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:10.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:10.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:10.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:11:10.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:10.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:11.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:11 smithi007 ceph-mon[32184]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:11.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:11 smithi007 ceph-mon[32184]: Updating smithi130:/etc/ceph/ceph.conf 2024-03-23T13:11:11.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:11 smithi007 ceph-mon[32184]: Deploying cephadm binary to smithi155 2024-03-23T13:11:11.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:11 smithi007 ceph-mon[32184]: Updating smithi130:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:12.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:12 smithi007 ceph-mon[32184]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-03-23T13:11:13.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:13 smithi007 ceph-mon[32184]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.127+0000 7fcc7b7fe700 1 -- 172.21.15.7:0/3920713600 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fcc840fd6d0 con 0x7fcc70038fe0 2024-03-23T13:11:14.120 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-03-23T13:11:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.129+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3920713600 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc70038fe0 msgr2=0x7fcc7003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.129+0000 7fcc8c27a700 1 --2- 172.21.15.7:0/3920713600 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc70038fe0 0x7fcc7003b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc80006fd0 tx=0x7fcc80006e40 comp rx=0 tx=0).stop 2024-03-23T13:11:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.130+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3920713600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc84106ac0 msgr2=0x7fcc840fca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.130+0000 7fcc8c27a700 1 --2- 172.21.15.7:0/3920713600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc84106ac0 0x7fcc840fca60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc74004f40 tx=0x7fcc74005020 comp rx=0 tx=0).stop 2024-03-23T13:11:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.130+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3920713600 shutdown_connections 2024-03-23T13:11:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.130+0000 7fcc8c27a700 1 --2- 172.21.15.7:0/3920713600 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc70038fe0 0x7fcc7003b4a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.130+0000 7fcc8c27a700 1 --2- 172.21.15.7:0/3920713600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc84106ac0 0x7fcc840fca60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.130+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3920713600 >> 172.21.15.7:0/3920713600 conn(0x7fcc840fc5b0 msgr2=0x7fcc841045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.130+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3920713600 shutdown_connections 2024-03-23T13:11:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:14.130+0000 7fcc8c27a700 1 -- 172.21.15.7:0/3920713600 wait complete. 2024-03-23T13:11:14.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-23T13:11:14.948 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:11:15.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:14 smithi007 ceph-mon[32184]: Updating smithi130:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T13:11:15.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:15.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:15.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:15.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:15.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:15.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:15.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:15 smithi007 ceph-mon[32184]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:15.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:15 smithi007 ceph-mon[32184]: Added host smithi155 2024-03-23T13:11:15.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:15 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.512+0000 7faeff0d8700 1 -- 172.21.15.7:0/2407740519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106ac0 msgr2=0x7faef8106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.512+0000 7faeff0d8700 1 --2- 172.21.15.7:0/2407740519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106ac0 0x7faef8106ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faef000b0a0 tx=0x7faef000b3b0 comp rx=0 tx=0).stop 2024-03-23T13:11:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.512+0000 7faeff0d8700 1 -- 172.21.15.7:0/2407740519 shutdown_connections 2024-03-23T13:11:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.512+0000 7faeff0d8700 1 --2- 172.21.15.7:0/2407740519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106ac0 0x7faef8106ea0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.512+0000 7faeff0d8700 1 -- 172.21.15.7:0/2407740519 >> 172.21.15.7:0/2407740519 conn(0x7faef80fc5b0 msgr2=0x7faef80fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.512+0000 7faeff0d8700 1 -- 172.21.15.7:0/2407740519 shutdown_connections 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.512+0000 7faeff0d8700 1 -- 172.21.15.7:0/2407740519 wait complete. 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.512+0000 7faeff0d8700 1 Processor -- start 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.512+0000 7faeff0d8700 1 -- start start 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faeff0d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106ac0 0x7faef8196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faeff0d8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faef806ac80 con 0x7faef8106ac0 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faefce74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106ac0 0x7faef8196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faefce74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106ac0 0x7faef8196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42362/0 (socket says 172.21.15.7:42362) 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faefce74700 1 -- 172.21.15.7:0/2287023489 learned_addr learned my addr 172.21.15.7:0/2287023489 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faefce74700 1 -- 172.21.15.7:0/2287023489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef0009d00 con 0x7faef8106ac0 2024-03-23T13:11:16.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faefce74700 1 --2- 172.21.15.7:0/2287023489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106ac0 0x7faef8196150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faef00062a0 tx=0x7faef000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faef5ffb700 1 -- 172.21.15.7:0/2287023489 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef000e040 con 0x7faef8106ac0 2024-03-23T13:11:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faef5ffb700 1 -- 172.21.15.7:0/2287023489 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef0003c10 con 0x7faef8106ac0 2024-03-23T13:11:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faeff0d8700 1 -- 172.21.15.7:0/2287023489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef8196690 con 0x7faef8106ac0 2024-03-23T13:11:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.513+0000 7faef5ffb700 1 -- 172.21.15.7:0/2287023489 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef001d600 con 0x7faef8106ac0 2024-03-23T13:11:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.514+0000 7faeff0d8700 1 -- 172.21.15.7:0/2287023489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef8196b10 con 0x7faef8106ac0 2024-03-23T13:11:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.515+0000 7faef5ffb700 1 -- 172.21.15.7:0/2287023489 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7faef0019070 con 0x7faef8106ac0 2024-03-23T13:11:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.515+0000 7faeff0d8700 1 -- 172.21.15.7:0/2287023489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef8102200 con 0x7faef8106ac0 2024-03-23T13:11:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.515+0000 7faef5ffb700 1 --2- 172.21.15.7:0/2287023489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faee0038fd0 0x7faee003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.515+0000 7faef5ffb700 1 -- 172.21.15.7:0/2287023489 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faef004d1a0 con 0x7faef8106ac0 2024-03-23T13:11:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.515+0000 7faef7fff700 1 --2- 172.21.15.7:0/2287023489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faee0038fd0 0x7faee003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.515+0000 7faef7fff700 1 --2- 172.21.15.7:0/2287023489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faee0038fd0 0x7faee003b490 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faeec006fd0 tx=0x7faeec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:16.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.520+0000 7faef5ffb700 1 -- 172.21.15.7:0/2287023489 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7faef00263c0 con 0x7faef8106ac0 2024-03-23T13:11:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.729+0000 7faeff0d8700 1 -- 172.21.15.7:0/2287023489 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faef806a570 con 0x7faee0038fd0 2024-03-23T13:11:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.730+0000 7faef5ffb700 1 -- 172.21.15.7:0/2287023489 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7faef806a570 con 0x7faee0038fd0 2024-03-23T13:11:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:11:16.722 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.130", "hostname": "smithi130", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-03-23T13:11:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.732+0000 7faeff0d8700 1 -- 172.21.15.7:0/2287023489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faee0038fd0 msgr2=0x7faee003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.732+0000 7faeff0d8700 1 --2- 172.21.15.7:0/2287023489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faee0038fd0 0x7faee003b490 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faeec006fd0 tx=0x7faeec006e40 comp rx=0 tx=0).stop 2024-03-23T13:11:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.732+0000 7faeff0d8700 1 -- 172.21.15.7:0/2287023489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106ac0 msgr2=0x7faef8196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.732+0000 7faeff0d8700 1 --2- 172.21.15.7:0/2287023489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106ac0 0x7faef8196150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faef00062a0 tx=0x7faef000bd40 comp rx=0 tx=0).stop 2024-03-23T13:11:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.732+0000 7faeff0d8700 1 -- 172.21.15.7:0/2287023489 shutdown_connections 2024-03-23T13:11:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.732+0000 7faeff0d8700 1 --2- 172.21.15.7:0/2287023489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faee0038fd0 0x7faee003b490 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.732+0000 7faeff0d8700 1 --2- 172.21.15.7:0/2287023489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faef8106ac0 0x7faef8196150 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.732+0000 7faeff0d8700 1 -- 172.21.15.7:0/2287023489 >> 172.21.15.7:0/2287023489 conn(0x7faef80fc5b0 msgr2=0x7faef81085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:16.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.732+0000 7faeff0d8700 1 -- 172.21.15.7:0/2287023489 shutdown_connections 2024-03-23T13:11:16.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:16.732+0000 7faeff0d8700 1 -- 172.21.15.7:0/2287023489 wait complete. 2024-03-23T13:11:17.410 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-23T13:11:17.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-23T13:11:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:11:18.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:17 smithi007 ceph-mon[32184]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:18.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:17 smithi007 ceph-mon[32184]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:11:18.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.119+0000 7f2429265700 1 -- 172.21.15.7:0/3404456360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2424101e40 msgr2=0x7f2424102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.119+0000 7f2429265700 1 --2- 172.21.15.7:0/3404456360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2424101e40 0x7f2424102220 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2414009910 tx=0x7f2414009c20 comp rx=0 tx=0).stop 2024-03-23T13:11:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.119+0000 7f2429265700 1 -- 172.21.15.7:0/3404456360 shutdown_connections 2024-03-23T13:11:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.119+0000 7f2429265700 1 --2- 172.21.15.7:0/3404456360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2424101e40 0x7f2424102220 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.119+0000 7f2429265700 1 -- 172.21.15.7:0/3404456360 >> 172.21.15.7:0/3404456360 conn(0x7f24240fd710 msgr2=0x7f24240ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.119+0000 7f2429265700 1 -- 172.21.15.7:0/3404456360 shutdown_connections 2024-03-23T13:11:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.119+0000 7f2429265700 1 -- 172.21.15.7:0/3404456360 wait complete. 2024-03-23T13:11:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.120+0000 7f2429265700 1 Processor -- start 2024-03-23T13:11:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.120+0000 7f2429265700 1 -- start start 2024-03-23T13:11:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.120+0000 7f2429265700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2424101e40 0x7f2424197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.120+0000 7f2429265700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2424197ac0 con 0x7f2424101e40 2024-03-23T13:11:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.120+0000 7f2422ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2424101e40 0x7f2424197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.120+0000 7f2422ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2424101e40 0x7f2424197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42380/0 (socket says 172.21.15.7:42380) 2024-03-23T13:11:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.120+0000 7f2422ffd700 1 -- 172.21.15.7:0/2712482513 learned_addr learned my addr 172.21.15.7:0/2712482513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:11:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.121+0000 7f2422ffd700 1 -- 172.21.15.7:0/2712482513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24140095f0 con 0x7f2424101e40 2024-03-23T13:11:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.121+0000 7f2422ffd700 1 --2- 172.21.15.7:0/2712482513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2424101e40 0x7f2424197580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2414004f40 tx=0x7f2414005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.121+0000 7f240bfff700 1 -- 172.21.15.7:0/2712482513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241401d070 con 0x7f2424101e40 2024-03-23T13:11:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.121+0000 7f240bfff700 1 -- 172.21.15.7:0/2712482513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241400bc30 con 0x7f2424101e40 2024-03-23T13:11:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.121+0000 7f2429265700 1 -- 172.21.15.7:0/2712482513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2424197cc0 con 0x7f2424101e40 2024-03-23T13:11:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.121+0000 7f240bfff700 1 -- 172.21.15.7:0/2712482513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2414003970 con 0x7f2424101e40 2024-03-23T13:11:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.121+0000 7f2429265700 1 -- 172.21.15.7:0/2712482513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24241980e0 con 0x7f2424101e40 2024-03-23T13:11:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.122+0000 7f240bfff700 1 -- 172.21.15.7:0/2712482513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7f2414003c30 con 0x7f2424101e40 2024-03-23T13:11:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.122+0000 7f2429265700 1 -- 172.21.15.7:0/2712482513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2424103320 con 0x7f2424101e40 2024-03-23T13:11:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.122+0000 7f240bfff700 1 --2- 172.21.15.7:0/2712482513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f240c038f80 0x7f240c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.122+0000 7f240bfff700 1 -- 172.21.15.7:0/2712482513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f241404cde0 con 0x7f2424101e40 2024-03-23T13:11:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.122+0000 7f24227fc700 1 --2- 172.21.15.7:0/2712482513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f240c038f80 0x7f240c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.123+0000 7f24227fc700 1 --2- 172.21.15.7:0/2712482513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f240c038f80 0x7f240c03b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2418006fd0 tx=0x7f2418006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.127+0000 7f240bfff700 1 -- 172.21.15.7:0/2712482513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2414029950 con 0x7f2424101e40 2024-03-23T13:11:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.338+0000 7f2429265700 1 -- 172.21.15.7:0/2712482513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f242410c070 con 0x7f2424101e40 2024-03-23T13:11:19.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.833+0000 7f240bfff700 1 -- 172.21.15.7:0/2712482513 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f2414026020 con 0x7f2424101e40 2024-03-23T13:11:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.835+0000 7f2429265700 1 -- 172.21.15.7:0/2712482513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f240c038f80 msgr2=0x7f240c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.835+0000 7f2429265700 1 --2- 172.21.15.7:0/2712482513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f240c038f80 0x7f240c03b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2418006fd0 tx=0x7f2418006e40 comp rx=0 tx=0).stop 2024-03-23T13:11:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.835+0000 7f2429265700 1 -- 172.21.15.7:0/2712482513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2424101e40 msgr2=0x7f2424197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.835+0000 7f2429265700 1 --2- 172.21.15.7:0/2712482513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2424101e40 0x7f2424197580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2414004f40 tx=0x7f2414005020 comp rx=0 tx=0).stop 2024-03-23T13:11:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.835+0000 7f2429265700 1 -- 172.21.15.7:0/2712482513 shutdown_connections 2024-03-23T13:11:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.835+0000 7f2429265700 1 --2- 172.21.15.7:0/2712482513 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f240c038f80 0x7f240c03b440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.835+0000 7f2429265700 1 --2- 172.21.15.7:0/2712482513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2424101e40 0x7f2424197580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.835+0000 7f2429265700 1 -- 172.21.15.7:0/2712482513 >> 172.21.15.7:0/2712482513 conn(0x7f24240fd710 msgr2=0x7f2424103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.836+0000 7f2429265700 1 -- 172.21.15.7:0/2712482513 shutdown_connections 2024-03-23T13:11:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:19.836+0000 7f2429265700 1 -- 172.21.15.7:0/2712482513 wait complete. 2024-03-23T13:11:19.829 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-03-23T13:11:20.153 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:19 smithi007 ceph-mon[32184]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:20.153 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:19 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2712482513' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-23T13:11:20.444 INFO:tasks.cephadm:Adding mon.a on smithi007 2024-03-23T13:11:20.444 INFO:tasks.cephadm:Adding mon.b on smithi130 2024-03-23T13:11:20.444 INFO:tasks.cephadm:Adding mon.c on smithi155 2024-03-23T13:11:20.444 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi007:172.21.15.7=a;smithi130:172.21.15.130=b;smithi155:172.21.15.155=c' 2024-03-23T13:11:20.676 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:21.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2712482513' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-23T13:11:21.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: Updating smithi155:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-23T13:11:21.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:20 smithi007 ceph-mon[32184]: Updating smithi155:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T13:11:22.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.183+0000 7fbd968b5700 1 -- 172.21.15.155:0/2800646721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd90101e40 msgr2=0x7fbd90102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:22.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.183+0000 7fbd968b5700 1 --2- 172.21.15.155:0/2800646721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd90101e40 0x7fbd90102220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009910 tx=0x7fbd80009c20 comp rx=0 tx=0).stop 2024-03-23T13:11:22.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.183+0000 7fbd968b5700 1 -- 172.21.15.155:0/2800646721 shutdown_connections 2024-03-23T13:11:22.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.183+0000 7fbd968b5700 1 --2- 172.21.15.155:0/2800646721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd90101e40 0x7fbd90102220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:22.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.183+0000 7fbd968b5700 1 -- 172.21.15.155:0/2800646721 >> 172.21.15.155:0/2800646721 conn(0x7fbd900fd6f0 msgr2=0x7fbd900ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:22.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.183+0000 7fbd968b5700 1 -- 172.21.15.155:0/2800646721 shutdown_connections 2024-03-23T13:11:22.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.184+0000 7fbd968b5700 1 -- 172.21.15.155:0/2800646721 wait complete. 2024-03-23T13:11:22.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.184+0000 7fbd968b5700 1 Processor -- start 2024-03-23T13:11:22.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.184+0000 7fbd968b5700 1 -- start start 2024-03-23T13:11:22.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.184+0000 7fbd968b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd90101e40 0x7fbd90197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:22.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.184+0000 7fbd968b5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd90197aa0 con 0x7fbd90101e40 2024-03-23T13:11:22.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.185+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd90101e40 0x7fbd90197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:22.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.185+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd90101e40 0x7fbd90197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.155:41944/0 (socket says 172.21.15.155:41944) 2024-03-23T13:11:22.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.185+0000 7fbd8ffff700 1 -- 172.21.15.155:0/1902634382 learned_addr learned my addr 172.21.15.155:0/1902634382 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-23T13:11:22.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.185+0000 7fbd8ffff700 1 -- 172.21.15.155:0/1902634382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd800095f0 con 0x7fbd90101e40 2024-03-23T13:11:22.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.185+0000 7fbd8ffff700 1 --2- 172.21.15.155:0/1902634382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd90101e40 0x7fbd90197560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbd80006010 tx=0x7fbd80004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:22.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.185+0000 7fbd8d7fa700 1 -- 172.21.15.155:0/1902634382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd8001d070 con 0x7fbd90101e40 2024-03-23T13:11:22.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.186+0000 7fbd8d7fa700 1 -- 172.21.15.155:0/1902634382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd8000bc30 con 0x7fbd90101e40 2024-03-23T13:11:22.179 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.186+0000 7fbd8d7fa700 1 -- 172.21.15.155:0/1902634382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd80003930 con 0x7fbd90101e40 2024-03-23T13:11:22.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.186+0000 7fbd968b5700 1 -- 172.21.15.155:0/1902634382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd90197ca0 con 0x7fbd90101e40 2024-03-23T13:11:22.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.186+0000 7fbd968b5700 1 -- 172.21.15.155:0/1902634382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd901980c0 con 0x7fbd90101e40 2024-03-23T13:11:22.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.187+0000 7fbd968b5700 1 -- 172.21.15.155:0/1902634382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd90063830 con 0x7fbd90101e40 2024-03-23T13:11:22.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.187+0000 7fbd8d7fa700 1 -- 172.21.15.155:0/1902634382 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7fbd80003ae0 con 0x7fbd90101e40 2024-03-23T13:11:22.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.187+0000 7fbd8d7fa700 1 --2- 172.21.15.155:0/1902634382 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd78038fe0 0x7fbd7803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:22.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.187+0000 7fbd8d7fa700 1 -- 172.21.15.155:0/1902634382 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbd8004cd70 con 0x7fbd90101e40 2024-03-23T13:11:22.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.187+0000 7fbd8f7fe700 1 --2- 172.21.15.155:0/1902634382 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd78038fe0 0x7fbd7803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:22.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.188+0000 7fbd8f7fe700 1 --2- 172.21.15.155:0/1902634382 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd78038fe0 0x7fbd7803b4a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbd84006fd0 tx=0x7fbd84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:22.184 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.193+0000 7fbd8d7fa700 1 -- 172.21.15.155:0/1902634382 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fbd80029540 con 0x7fbd90101e40 2024-03-23T13:11:22.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:22.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:22.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:22.392 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.399+0000 7fbd968b5700 1 -- 172.21.15.155:0/1902634382 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi130:172.21.15.130=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}) -- 0x7fbd9010c150 con 0x7fbd78038fe0 2024-03-23T13:11:22.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.407+0000 7fbd8d7fa700 1 -- 172.21.15.155:0/1902634382 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbd9010c150 con 0x7fbd78038fe0 2024-03-23T13:11:22.399 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mon update... 2024-03-23T13:11:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.409+0000 7fbd968b5700 1 -- 172.21.15.155:0/1902634382 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd78038fe0 msgr2=0x7fbd7803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.409+0000 7fbd968b5700 1 --2- 172.21.15.155:0/1902634382 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd78038fe0 0x7fbd7803b4a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbd84006fd0 tx=0x7fbd84006e40 comp rx=0 tx=0).stop 2024-03-23T13:11:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.409+0000 7fbd968b5700 1 -- 172.21.15.155:0/1902634382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd90101e40 msgr2=0x7fbd90197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.409+0000 7fbd968b5700 1 --2- 172.21.15.155:0/1902634382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd90101e40 0x7fbd90197560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbd80006010 tx=0x7fbd80004ff0 comp rx=0 tx=0).stop 2024-03-23T13:11:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.409+0000 7fbd968b5700 1 -- 172.21.15.155:0/1902634382 shutdown_connections 2024-03-23T13:11:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.409+0000 7fbd968b5700 1 --2- 172.21.15.155:0/1902634382 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd78038fe0 0x7fbd7803b4a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.409+0000 7fbd968b5700 1 --2- 172.21.15.155:0/1902634382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd90101e40 0x7fbd90197560 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.409+0000 7fbd968b5700 1 -- 172.21.15.155:0/1902634382 >> 172.21.15.155:0/1902634382 conn(0x7fbd900fd6f0 msgr2=0x7fbd90103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.409+0000 7fbd968b5700 1 -- 172.21.15.155:0/1902634382 shutdown_connections 2024-03-23T13:11:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:22.409+0000 7fbd968b5700 1 -- 172.21.15.155:0/1902634382 wait complete. 2024-03-23T13:11:22.913 DEBUG:teuthology.orchestra.run.smithi130:mon.b> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.b.service 2024-03-23T13:11:22.916 DEBUG:teuthology.orchestra.run.smithi155:mon.c> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.c.service 2024-03-23T13:11:22.918 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T13:11:22.918 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T13:11:23.612 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:23.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:23.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi130:172.21.15.130=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi130:172.21.15.130=b;smithi155:172.21.15.155=c;count:3 2024-03-23T13:11:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:23 smithi007 ceph-mon[32184]: Deploying daemon mon.c on smithi155 2024-03-23T13:11:25.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:25 smithi007 ceph-mon[32184]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:25.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:25 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/2839801426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T13:11:25.929 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.964+0000 7f5195b9b700 1 -- 172.21.15.155:0/3135894776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5190102d80 msgr2=0x7f5190103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:25.929 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.964+0000 7f5195b9b700 1 --2- 172.21.15.155:0/3135894776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5190102d80 0x7f5190103160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5180009910 tx=0x7f5180009c20 comp rx=0 tx=0).stop 2024-03-23T13:11:25.929 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.964+0000 7f5195b9b700 1 -- 172.21.15.155:0/3135894776 shutdown_connections 2024-03-23T13:11:25.929 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.964+0000 7f5195b9b700 1 --2- 172.21.15.155:0/3135894776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5190102d80 0x7f5190103160 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:25.929 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.964+0000 7f5195b9b700 1 -- 172.21.15.155:0/3135894776 >> 172.21.15.155:0/3135894776 conn(0x7f51900fc5b0 msgr2=0x7f51900fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:25.929 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.964+0000 7f5195b9b700 1 -- 172.21.15.155:0/3135894776 shutdown_connections 2024-03-23T13:11:25.929 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.964+0000 7f5195b9b700 1 -- 172.21.15.155:0/3135894776 wait complete. 2024-03-23T13:11:25.929 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.964+0000 7f5195b9b700 1 Processor -- start 2024-03-23T13:11:25.929 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.965+0000 7f5195b9b700 1 -- start start 2024-03-23T13:11:25.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.965+0000 7f5195b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5190102d80 0x7f5190199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:25.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.965+0000 7f5195b9b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5190199c90 con 0x7f5190102d80 2024-03-23T13:11:25.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.965+0000 7f518f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5190102d80 0x7f5190199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:25.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.965+0000 7f518f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5190102d80 0x7f5190199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.155:41954/0 (socket says 172.21.15.155:41954) 2024-03-23T13:11:25.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.965+0000 7f518f7fe700 1 -- 172.21.15.155:0/2839801426 learned_addr learned my addr 172.21.15.155:0/2839801426 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-23T13:11:25.930 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:11:25.930 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","modified":"2024-03-23T13:09:36.205070Z","created":"2024-03-23T13:09:36.205070Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T13:11:25.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.965+0000 7f518f7fe700 1 -- 172.21.15.155:0/2839801426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51800095f0 con 0x7f5190102d80 2024-03-23T13:11:25.930 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.966+0000 7f518f7fe700 1 --2- 172.21.15.155:0/2839801426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5190102d80 0x7f5190199750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5180005850 tx=0x7f5180004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.966+0000 7f518cff9700 1 -- 172.21.15.155:0/2839801426 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f518001d070 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.966+0000 7f5195b9b700 1 -- 172.21.15.155:0/2839801426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5190199e90 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.966+0000 7f518cff9700 1 -- 172.21.15.155:0/2839801426 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f518000bc30 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.966+0000 7f518cff9700 1 -- 172.21.15.155:0/2839801426 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5180003a20 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.966+0000 7f5195b9b700 1 -- 172.21.15.155:0/2839801426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519019a2b0 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.967+0000 7f5195b9b700 1 -- 172.21.15.155:0/2839801426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5190063830 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.967+0000 7f518cff9700 1 -- 172.21.15.155:0/2839801426 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7f5180003c90 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.967+0000 7f518cff9700 1 --2- 172.21.15.155:0/2839801426 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f517c038f80 0x7f517c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.967+0000 7f518cff9700 1 -- 172.21.15.155:0/2839801426 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f518004cd60 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.967+0000 7f518effd700 1 --2- 172.21.15.155:0/2839801426 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f517c038f80 0x7f517c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.968+0000 7f518effd700 1 --2- 172.21.15.155:0/2839801426 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f517c038f80 0x7f517c03b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5178006fd0 tx=0x7f5178006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:24.972+0000 7f518cff9700 1 -- 172.21.15.155:0/2839801426 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f518000b6f0 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.249+0000 7f5195b9b700 1 -- 172.21.15.155:0/2839801426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f519010c090 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.250+0000 7f518cff9700 1 -- 172.21.15.155:0/2839801426 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f5180026070 con 0x7f5190102d80 2024-03-23T13:11:25.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.252+0000 7f5195b9b700 1 -- 172.21.15.155:0/2839801426 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f517c038f80 msgr2=0x7f517c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:25.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.252+0000 7f5195b9b700 1 --2- 172.21.15.155:0/2839801426 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f517c038f80 0x7f517c03b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5178006fd0 tx=0x7f5178006e40 comp rx=0 tx=0).stop 2024-03-23T13:11:25.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.252+0000 7f5195b9b700 1 -- 172.21.15.155:0/2839801426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5190102d80 msgr2=0x7f5190199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:25.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.252+0000 7f5195b9b700 1 --2- 172.21.15.155:0/2839801426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5190102d80 0x7f5190199750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5180005850 tx=0x7f5180004ff0 comp rx=0 tx=0).stop 2024-03-23T13:11:25.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.252+0000 7f5195b9b700 1 -- 172.21.15.155:0/2839801426 shutdown_connections 2024-03-23T13:11:25.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.252+0000 7f5195b9b700 1 --2- 172.21.15.155:0/2839801426 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f517c038f80 0x7f517c03b440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:25.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.252+0000 7f5195b9b700 1 --2- 172.21.15.155:0/2839801426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5190102d80 0x7f5190199750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:25.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.252+0000 7f5195b9b700 1 -- 172.21.15.155:0/2839801426 >> 172.21.15.155:0/2839801426 conn(0x7f51900fc5b0 msgr2=0x7f5190107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:25.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.252+0000 7f5195b9b700 1 -- 172.21.15.155:0/2839801426 shutdown_connections 2024-03-23T13:11:25.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:25.252+0000 7f5195b9b700 1 -- 172.21.15.155:0/2839801426 wait complete. 2024-03-23T13:11:25.932 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-03-23T13:11:27.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:27 smithi007 ceph-mon[32184]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:28.056 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T13:11:28.056 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T13:11:28.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:27 smithi155 systemd[1]: Started Ceph mon.c for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:11:28.289 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:11:32.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.939+0000 7f17e691d700 1 -- 172.21.15.155:0/4244890015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c80057d0 msgr2=0x7f17c8007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:32.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.939+0000 7f17e691d700 1 --2- 172.21.15.155:0/4244890015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c80057d0 0x7f17c8007c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f17e0100b30 tx=0x7f17d0013e90 comp rx=0 tx=0).stop 2024-03-23T13:11:32.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.939+0000 7f17e691d700 1 -- 172.21.15.155:0/4244890015 shutdown_connections 2024-03-23T13:11:32.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.939+0000 7f17e691d700 1 --2- 172.21.15.155:0/4244890015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c80057d0 0x7f17c8007c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:32.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.939+0000 7f17e691d700 1 -- 172.21.15.155:0/4244890015 >> 172.21.15.155:0/4244890015 conn(0x7f17e00fc710 msgr2=0x7f17e00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:32.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.939+0000 7f17e691d700 1 -- 172.21.15.155:0/4244890015 shutdown_connections 2024-03-23T13:11:32.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.939+0000 7f17e691d700 1 -- 172.21.15.155:0/4244890015 wait complete. 2024-03-23T13:11:32.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.940+0000 7f17e691d700 1 Processor -- start 2024-03-23T13:11:32.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.940+0000 7f17e691d700 1 -- start start 2024-03-23T13:11:32.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17e691d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c80057d0 0x7f17e019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:32.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17e691d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e019fbb0 0x7f17e01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:32.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17e691d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01a00d0 con 0x7f17c80057d0 2024-03-23T13:11:32.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17e691d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0199d20 con 0x7f17e019fbb0 2024-03-23T13:11:32.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17df7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e019fbb0 0x7f17e01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:32.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17df7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e019fbb0 0x7f17e01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:41420/0 (socket says 172.21.15.155:41420) 2024-03-23T13:11:32.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17df7fe700 1 -- 172.21.15.155:0/332636392 learned_addr learned my addr 172.21.15.155:0/332636392 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17dffff700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c80057d0 0x7f17e019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17df7fe700 1 -- 172.21.15.155:0/332636392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e019fbb0 msgr2=0x7f17e01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17df7fe700 1 -- 172.21.15.155:0/332636392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e019fbb0 msgr2=0x7f17e01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17df7fe700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e019fbb0 0x7f17e01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17df7fe700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e019fbb0 0x7f17e01997e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17dffff700 1 -- 172.21.15.155:0/332636392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e019fbb0 msgr2=0x7f17e01997e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17dffff700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e019fbb0 0x7f17e01997e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.941+0000 7f17dffff700 1 -- 172.21.15.155:0/332636392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d00138e0 con 0x7f17c80057d0 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.942+0000 7f17dffff700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c80057d0 0x7f17e019f670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f17d001c000 tx=0x7f17d0005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.942+0000 7f17dd7fa700 1 -- 172.21.15.155:0/332636392 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f17d0018070 con 0x7f17c80057d0 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.942+0000 7f17dd7fa700 1 -- 172.21.15.155:0/332636392 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d0005090 con 0x7f17c80057d0 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.942+0000 7f17e691d700 1 -- 172.21.15.155:0/332636392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e0199f50 con 0x7f17c80057d0 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.942+0000 7f17dd7fa700 1 -- 172.21.15.155:0/332636392 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f17d0003ad0 con 0x7f17c80057d0 2024-03-23T13:11:32.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.942+0000 7f17e691d700 1 -- 172.21.15.155:0/332636392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e019a4a0 con 0x7f17c80057d0 2024-03-23T13:11:32.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.943+0000 7f17dd7fa700 1 -- 172.21.15.155:0/332636392 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7f17d001c410 con 0x7f17c80057d0 2024-03-23T13:11:32.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.943+0000 7f17e691d700 1 -- 172.21.15.155:0/332636392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e019a710 con 0x7f17c80057d0 2024-03-23T13:11:32.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.943+0000 7f17dd7fa700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f17cc038d20 0x7f17cc03b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:32.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.943+0000 7f17dd7fa700 1 -- 172.21.15.155:0/332636392 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f17d004e9d0 con 0x7f17c80057d0 2024-03-23T13:11:32.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.943+0000 7f17df7fe700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f17cc038d20 0x7f17cc03b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:32.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.944+0000 7f17df7fe700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f17cc038d20 0x7f17cc03b1e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009180 tx=0x7f17d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:32.941 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:32.949+0000 7f17dd7fa700 1 -- 172.21.15.155:0/332636392 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f17d002d9a0 con 0x7f17c80057d0 2024-03-23T13:11:33.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.226+0000 7f17e691d700 1 -- 172.21.15.155:0/332636392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f17e019ac70 con 0x7f17c80057d0 2024-03-23T13:11:33.219 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:33.219 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: mon.a calling monitor election 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: mon.c calling monitor election 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: monmap epoch 2 2024-03-23T13:11:33.220 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: last_changed 2024-03-23T13:11:27.862379+0000 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: created 2024-03-23T13:09:36.205070+0000 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: min_mon_release 19 (squid) 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: election_strategy: 1 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: fsmap 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: mgrmap e13: x(active, since 51s) 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: overall HEALTH_OK 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:33.221 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:33.222 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:33.222 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:33.222 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.226+0000 7f17dd7fa700 1 -- 172.21.15.155:0/332636392 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f17d002d3b0 con 0x7f17c80057d0 2024-03-23T13:11:33.222 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:11:33.222 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":2,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","modified":"2024-03-23T13:11:27.862379Z","created":"2024-03-23T13:09:36.205070Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-23T13:11:33.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.228+0000 7f17e691d700 1 -- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f17cc038d20 msgr2=0x7f17cc03b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:33.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.228+0000 7f17e691d700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f17cc038d20 0x7f17cc03b1e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009180 tx=0x7f17d4008040 comp rx=0 tx=0).stop 2024-03-23T13:11:33.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.228+0000 7f17e691d700 1 -- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c80057d0 msgr2=0x7f17e019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:33.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.228+0000 7f17e691d700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c80057d0 0x7f17e019f670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f17d001c000 tx=0x7f17d0005340 comp rx=0 tx=0).stop 2024-03-23T13:11:33.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.229+0000 7f17e691d700 1 -- 172.21.15.155:0/332636392 shutdown_connections 2024-03-23T13:11:33.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.229+0000 7f17e691d700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f17cc038d20 0x7f17cc03b1e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:33.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.229+0000 7f17e691d700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c80057d0 0x7f17e019f670 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:33.223 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.229+0000 7f17e691d700 1 --2- 172.21.15.155:0/332636392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e019fbb0 0x7f17e01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:33.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.229+0000 7f17e691d700 1 -- 172.21.15.155:0/332636392 >> 172.21.15.155:0/332636392 conn(0x7f17e00fc710 msgr2=0x7f17e00fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:33.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.229+0000 7f17e691d700 1 -- 172.21.15.155:0/332636392 shutdown_connections 2024-03-23T13:11:33.224 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:33.229+0000 7f17e691d700 1 -- 172.21.15.155:0/332636392 wait complete. 2024-03-23T13:11:33.224 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 2 2024-03-23T13:11:33.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:33.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:11:33.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: mon.a calling monitor election 2024-03-23T13:11:33.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:33.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: mon.c calling monitor election 2024-03-23T13:11:33.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: monmap epoch 2 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: last_changed 2024-03-23T13:11:27.862379+0000 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: created 2024-03-23T13:09:36.205070+0000 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: min_mon_release 19 (squid) 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: election_strategy: 1 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: fsmap 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: mgrmap e13: x(active, since 51s) 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: overall HEALTH_OK 2024-03-23T13:11:33.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:33.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:33.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:33.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:33.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:34.280 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:33 smithi155 ceph-mon[36157]: Deploying daemon mon.b on smithi130 2024-03-23T13:11:34.280 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:33 smithi155 ceph-mon[36157]: from='client.? 172.21.15.155:0/332636392' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T13:11:34.281 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:33 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:34.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:33 smithi007 ceph-mon[32184]: Deploying daemon mon.b on smithi130 2024-03-23T13:11:34.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:33 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/332636392' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T13:11:34.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:33 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:34.812 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T13:11:34.812 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T13:11:35.057 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:11:35.179 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:34 smithi155 ceph-mon[36157]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:35.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:34 smithi007 ceph-mon[32184]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.669 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 podman[36205]: 2024-03-23 13:11:36.653632504 +0000 UTC m=+0.345003757 container start 42a461ed95f7fefd5b0ab3ac10bffeada1e34618f438e3ba634846b6e79cd078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-b, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-23T13:11:36.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.684+0000 7fe5d14fe700 1 -- 172.21.15.155:0/4197552520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5cc106d00 msgr2=0x7fe5cc1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:36.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.684+0000 7fe5d14fe700 1 --2- 172.21.15.155:0/4197552520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5cc106d00 0x7fe5cc1070e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009910 tx=0x7fe5bc009c20 comp rx=0 tx=0).stop 2024-03-23T13:11:36.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.684+0000 7fe5d14fe700 1 -- 172.21.15.155:0/4197552520 shutdown_connections 2024-03-23T13:11:36.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.684+0000 7fe5d14fe700 1 --2- 172.21.15.155:0/4197552520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5cc106d00 0x7fe5cc1070e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:36.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.684+0000 7fe5d14fe700 1 -- 172.21.15.155:0/4197552520 >> 172.21.15.155:0/4197552520 conn(0x7fe5cc0772f0 msgr2=0x7fe5cc077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:36.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.684+0000 7fe5d14fe700 1 -- 172.21.15.155:0/4197552520 shutdown_connections 2024-03-23T13:11:36.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.684+0000 7fe5d14fe700 1 -- 172.21.15.155:0/4197552520 wait complete. 2024-03-23T13:11:36.678 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.685+0000 7fe5d14fe700 1 Processor -- start 2024-03-23T13:11:36.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.685+0000 7fe5d14fe700 1 -- start start 2024-03-23T13:11:36.679 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.685+0000 7fe5d14fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5cc106d00 0x7fe5cc198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.685+0000 7fe5d14fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5cc198b40 0x7fe5cc198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.685+0000 7fe5d14fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc101290 con 0x7fe5cc198b40 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.685+0000 7fe5d14fe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc1a1670 con 0x7fe5cc106d00 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.685+0000 7fe5c27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5cc198b40 0x7fe5cc198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.685+0000 7fe5c27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5cc198b40 0x7fe5cc198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.155:33536/0 (socket says 172.21.15.155:33536) 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.685+0000 7fe5c27fc700 1 -- 172.21.15.155:0/3069244884 learned_addr learned my addr 172.21.15.155:0/3069244884 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.685+0000 7fe5caffd700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5cc106d00 0x7fe5cc198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5caffd700 1 -- 172.21.15.155:0/3069244884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5cc106d00 msgr2=0x7fe5cc198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5caffd700 1 -- 172.21.15.155:0/3069244884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5cc106d00 msgr2=0x7fe5cc198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5caffd700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5cc106d00 0x7fe5cc198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5caffd700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5cc106d00 0x7fe5cc198600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5c27fc700 1 -- 172.21.15.155:0/3069244884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5cc106d00 msgr2=0x7fe5cc198600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T13:11:36.680 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5c27fc700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5cc106d00 0x7fe5cc198600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:36.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5c27fc700 1 -- 172.21.15.155:0/3069244884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc0095f0 con 0x7fe5cc198b40 2024-03-23T13:11:36.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5c27fc700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5cc198b40 0x7fe5cc198fc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400b700 tx=0x7fe5b400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:36.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5c8ff9700 1 -- 172.21.15.155:0/3069244884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe5b4011630 con 0x7fe5cc198b40 2024-03-23T13:11:36.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5d14fe700 1 -- 172.21.15.155:0/3069244884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc1a1900 con 0x7fe5cc198b40 2024-03-23T13:11:36.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5c8ff9700 1 -- 172.21.15.155:0/3069244884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b4011c70 con 0x7fe5cc198b40 2024-03-23T13:11:36.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5c8ff9700 1 -- 172.21.15.155:0/3069244884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe5b4010de0 con 0x7fe5cc198b40 2024-03-23T13:11:36.681 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.686+0000 7fe5d14fe700 1 -- 172.21.15.155:0/3069244884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc1a1e50 con 0x7fe5cc198b40 2024-03-23T13:11:36.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.687+0000 7fe5c8ff9700 1 -- 172.21.15.155:0/3069244884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7fe5b401a400 con 0x7fe5cc198b40 2024-03-23T13:11:36.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.687+0000 7fe5d14fe700 1 -- 172.21.15.155:0/3069244884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5cc063830 con 0x7fe5cc198b40 2024-03-23T13:11:36.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.687+0000 7fe5c8ff9700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5b8038d20 0x7fe5b803b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:36.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.688+0000 7fe5c8ff9700 1 -- 172.21.15.155:0/3069244884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe5b404b830 con 0x7fe5cc198b40 2024-03-23T13:11:36.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.688+0000 7fe5caffd700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5b8038d20 0x7fe5b803b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:36.682 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.688+0000 7fe5caffd700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5b8038d20 0x7fe5b803b1e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005b40 tx=0x7fe5bc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:36.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.693+0000 7fe5c8ff9700 1 -- 172.21.15.155:0/3069244884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fe5b40178b0 con 0x7fe5cc198b40 2024-03-23T13:11:36.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.706+0000 7fe5c8ff9700 1 -- 172.21.15.155:0/3069244884 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40178b0 con 0x7fe5cc198b40 2024-03-23T13:11:36.937 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev), process ceph-mon, pid 2 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pidfile_write: ignore empty --pid-file 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: load: jerasure load: lrc 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: RocksDB version: 7.9.2 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Git sha 0 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Compile date 2024-03-23 03:28:38 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: DB SUMMARY 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: DB Session ID: QFNIY838C8MF0N43GJ2Y 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: CURRENT file: CURRENT 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T13:11:36.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.error_if_exists: 0 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.create_if_missing: 0 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.paranoid_checks: 1 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.env: 0x55e60695bc20 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.fs: PosixFileSystem 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.info_log: 0x55e60757bac0 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.statistics: (nil) 2024-03-23T13:11:36.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.use_fsync: 0 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_log_file_size: 0 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.allow_fallocate: 1 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.use_direct_reads: 0 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.db_log_dir: 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.wal_dir: 2024-03-23T13:11:36.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T13:11:36.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T13:11:36.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T13:11:36.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.write_buffer_manager: 0x55e60754e1e0 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.unordered_write: 0 2024-03-23T13:11:36.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.row_cache: None 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.wal_filter: None 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.two_write_queues: 0 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.wal_compression: 0 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.atomic_flush: 0 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.log_readahead_size: 0 2024-03-23T13:11:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_background_jobs: 2 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_background_compactions: -1 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_subcompactions: 1 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T13:11:36.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_open_files: -1 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_background_flushes: -1 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Compression algorithms supported: 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: kZSTD supported: 0 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: kXpressCompression supported: 0 2024-03-23T13:11:36.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: kZlibCompression supported: 1 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: kSnappyCompression supported: 1 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: kLZ4Compression supported: 1 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: kBZip2Compression supported: 0 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.merge_operator: 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_filter: None 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T13:11:36.947 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e60757bbc0) 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: cache_index_and_filter_blocks: 1 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: pin_top_level_index_and_filter: 1 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: index_type: 0 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: data_block_index_type: 0 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: index_shortening: 1 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: checksum: 4 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: no_block_cache: 0 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_cache: 0x55e60757f090 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_cache_name: BinnedLRUCache 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_cache_options: 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: capacity : 536870912 2024-03-23T13:11:36.948 INFO:journalctl@ceph.mon.b.smithi130.stdout: num_shard_bits : 4 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: strict_capacity_limit : 0 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: high_pri_pool_ratio: 0.000 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_cache_compressed: (nil) 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: persistent_cache: (nil) 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_size: 4096 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_size_deviation: 10 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_restart_interval: 16 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: index_block_restart_interval: 1 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: metadata_block_size: 4096 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: partition_filters: 0 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: use_delta_encoding: 1 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: filter_policy: bloomfilter 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: whole_key_filtering: 1 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: verify_compression: 0 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: read_amp_bytes_per_bit: 0 2024-03-23T13:11:36.949 INFO:journalctl@ceph.mon.b.smithi130.stdout: format_version: 5 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout: enable_index_compression: 1 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout: block_align: 0 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout: max_auto_readahead_size: 262144 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout: prepopulate_block_cache: 0 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout: initial_auto_readahead_size: 8192 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout: num_file_reads_for_auto_readahead: 2 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compression: NoCompression 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T13:11:36.950 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.num_levels: 7 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T13:11:36.951 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T13:11:36.952 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.arena_block_size: 1048576 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T13:11:36.953 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.table_properties_collectors: 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.inplace_update_support: 0 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.bloom_locality: 0 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.max_successive_merges: 0 2024-03-23T13:11:36.954 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.ttl: 2592000 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.enable_blob_files: false 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.min_blob_size: 0 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T13:11:36.955 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.blob_file_starting_level: 0 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 57fb6659-6071-4411-821c-17ae42503b4d 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711199496683492, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711199496683913, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711199496, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "57fb6659-6071-4411-821c-17ae42503b4d", "db_session_id": "QFNIY838C8MF0N43GJ2Y", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711199496684002, "job": 1, "event": "recovery_finished"} 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e607670000 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: DB pointer 0x55e60765a000 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-23T13:11:36.956 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: ** DB Stats ** 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: ** Compaction Stats [default] ** 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T13:11:36.957 INFO:journalctl@ceph.mon.b.smithi130.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: ** Compaction Stats [default] ** 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T13:11:36.958 INFO:journalctl@ceph.mon.b.smithi130.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout: Cumulative compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout: Interval compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout: Block cache BinnedLRUCache@0x55e60757f090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: using public_addr v2:172.21.15.130:0/0 -> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: starting mon.b rank -1 at public addrs [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] at bind addrs [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(???) e0 preinit fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).mds e1 new map 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-23T13:11:36.959 INFO:journalctl@ceph.mon.b.smithi130.stdout: e1 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout: legacy client fscid: -1 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout: 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout: No filesystems configured 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T13:11:36.960 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Updating smithi130:/etc/ceph/ceph.conf 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Deploying cephadm binary to smithi155 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Updating smithi130:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.961 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Updating smithi130:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Added host smithi155 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2712482513' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-23T13:11:36.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2712482513' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Updating smithi155:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Updating smithi155:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.963 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi130:172.21.15.130=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi130:172.21.15.130=b;smithi155:172.21.15.155=c;count:3 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Deploying daemon mon.c on smithi155 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/2839801426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T13:11:36.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.a calling monitor election 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.c calling monitor election 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: monmap epoch 2 2024-03-23T13:11:36.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: last_changed 2024-03-23T13:11:27.862379+0000 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: created 2024-03-23T13:09:36.205070+0000 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: min_mon_release 19 (squid) 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: election_strategy: 1 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: fsmap 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mgrmap e13: x(active, since 51s) 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: overall HEALTH_OK 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:36.966 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:36.967 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:36.967 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: Deploying daemon mon.b on smithi130 2024-03-23T13:11:36.967 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/332636392' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T13:11:36.967 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:36.967 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:36.967 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 ceph-mon[36225]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-23T13:11:36.967 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 bash[36205]: 42a461ed95f7fefd5b0ab3ac10bffeada1e34618f438e3ba634846b6e79cd078 2024-03-23T13:11:36.967 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:36 smithi130 systemd[1]: Started Ceph mon.b for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:11:36.967 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:36.966+0000 7fe5d14fe700 1 -- 172.21.15.155:0/3069244884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe5cc1008d0 con 0x7fe5cc198b40 2024-03-23T13:11:41.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.738+0000 7fe5c8ff9700 1 -- 172.21.15.155:0/3069244884 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7fe5b400c150 con 0x7fe5cc198b40 2024-03-23T13:11:41.732 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-23T13:11:41.732 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":3,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","modified":"2024-03-23T13:11:36.706371Z","created":"2024-03-23T13:09:36.205070Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-23T13:11:41.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.741+0000 7fe5d14fe700 1 -- 172.21.15.155:0/3069244884 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5b8038d20 msgr2=0x7fe5b803b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.741+0000 7fe5d14fe700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5b8038d20 0x7fe5b803b1e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005b40 tx=0x7fe5bc005a90 comp rx=0 tx=0).stop 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.741+0000 7fe5d14fe700 1 -- 172.21.15.155:0/3069244884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5cc198b40 msgr2=0x7fe5cc198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.741+0000 7fe5d14fe700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5cc198b40 0x7fe5cc198fc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400b700 tx=0x7fe5b400bac0 comp rx=0 tx=0).stop 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.741+0000 7fe5d14fe700 1 -- 172.21.15.155:0/3069244884 shutdown_connections 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.741+0000 7fe5d14fe700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5b8038d20 0x7fe5b803b1e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.741+0000 7fe5d14fe700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5cc106d00 0x7fe5cc198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.741+0000 7fe5d14fe700 1 --2- 172.21.15.155:0/3069244884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5cc198b40 0x7fe5cc198fc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.741+0000 7fe5d14fe700 1 -- 172.21.15.155:0/3069244884 >> 172.21.15.155:0/3069244884 conn(0x7fe5cc0772f0 msgr2=0x7fe5cc0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.742+0000 7fe5d14fe700 1 -- 172.21.15.155:0/3069244884 shutdown_connections 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:41.742+0000 7fe5d14fe700 1 -- 172.21.15.155:0/3069244884 wait complete. 2024-03-23T13:11:41.735 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 3 2024-03-23T13:11:42.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:42.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:11:42.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: mon.c calling monitor election 2024-03-23T13:11:42.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: mon.a calling monitor election 2024-03-23T13:11:42.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:42.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='client.? 172.21.15.155:0/3069244884' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: monmap epoch 3 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: last_changed 2024-03-23T13:11:36.706371+0000 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: created 2024-03-23T13:09:36.205070+0000 2024-03-23T13:11:42.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: min_mon_release 19 (squid) 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: election_strategy: 1 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: fsmap 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: mgrmap e13: x(active, since 59s) 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: overall HEALTH_OK 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:42.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:42.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:42.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:11:42.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: mon.c calling monitor election 2024-03-23T13:11:42.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: mon.a calling monitor election 2024-03-23T13:11:42.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:42.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/3069244884' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T13:11:42.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:42.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: monmap epoch 3 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: last_changed 2024-03-23T13:11:36.706371+0000 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: created 2024-03-23T13:09:36.205070+0000 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: min_mon_release 19 (squid) 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: election_strategy: 1 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-03-23T13:11:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-03-23T13:11:42.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: fsmap 2024-03-23T13:11:42.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:42.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: mgrmap e13: x(active, since 59s) 2024-03-23T13:11:42.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: overall HEALTH_OK 2024-03-23T13:11:42.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:42.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:42.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:42.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:42.324 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-23T13:11:42.325 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-23T13:11:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:11:43.006 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:42 smithi007 ceph-mon[32184]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:43.006 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:43.006 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:43.006 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:43.030 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:42 smithi155 ceph-mon[36157]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:43.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:43.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:43.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: mon.c calling monitor election 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: mon.a calling monitor election 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/3069244884' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:44.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: monmap epoch 3 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: last_changed 2024-03-23T13:11:36.706371+0000 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: created 2024-03-23T13:09:36.205070+0000 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: min_mon_release 19 (squid) 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: election_strategy: 1 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-03-23T13:11:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: fsmap 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: mgrmap e13: x(active, since 59s) 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: overall HEALTH_OK 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:44.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:44.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.225+0000 7fed4c6a3700 1 -- 172.21.15.7:0/1452499264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed44101e40 msgr2=0x7fed44102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.225+0000 7fed4c6a3700 1 --2- 172.21.15.7:0/1452499264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed44101e40 0x7fed44102220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fed40009910 tx=0x7fed40009c20 comp rx=0 tx=0).stop 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.225+0000 7fed4c6a3700 1 -- 172.21.15.7:0/1452499264 shutdown_connections 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.225+0000 7fed4c6a3700 1 --2- 172.21.15.7:0/1452499264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed44101e40 0x7fed44102220 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.225+0000 7fed4c6a3700 1 -- 172.21.15.7:0/1452499264 >> 172.21.15.7:0/1452499264 conn(0x7fed440fd6f0 msgr2=0x7fed440ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.226+0000 7fed4c6a3700 1 -- 172.21.15.7:0/1452499264 shutdown_connections 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.226+0000 7fed4c6a3700 1 -- 172.21.15.7:0/1452499264 wait complete. 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.226+0000 7fed4c6a3700 1 Processor -- start 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.226+0000 7fed4c6a3700 1 -- start start 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.226+0000 7fed4c6a3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed44101e40 0x7fed4419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.226+0000 7fed4c6a3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed4419fc00 0x7fed44199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.226+0000 7fed4c6a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed44199d80 0x7fed4419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.226+0000 7fed4c6a3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed441a0270 con 0x7fed44199d80 2024-03-23T13:11:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.226+0000 7fed4c6a3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fed441a6520 con 0x7fed44101e40 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.226+0000 7fed4c6a3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed441a6690 con 0x7fed4419fc00 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4ac40700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed44199d80 0x7fed4419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4ac40700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed44199d80 0x7fed4419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57502/0 (socket says 172.21.15.7:57502) 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4ac40700 1 -- 172.21.15.7:0/206056770 learned_addr learned my addr 172.21.15.7:0/206056770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed49c3e700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed4419fc00 0x7fed44199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4a43f700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed44101e40 0x7fed4419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4ac40700 1 -- 172.21.15.7:0/206056770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed4419fc00 msgr2=0x7fed44199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4ac40700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed4419fc00 0x7fed44199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4ac40700 1 -- 172.21.15.7:0/206056770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed44101e40 msgr2=0x7fed4419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4ac40700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed44101e40 0x7fed4419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4ac40700 1 -- 172.21.15.7:0/206056770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed400095f0 con 0x7fed44199d80 2024-03-23T13:11:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4ac40700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed44199d80 0x7fed4419a230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00ba50 tx=0x7fed3c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:44.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed3b7fe700 1 -- 172.21.15.7:0/206056770 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c00d8e0 con 0x7fed44199d80 2024-03-23T13:11:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed3b7fe700 1 -- 172.21.15.7:0/206056770 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c009930 con 0x7fed44199d80 2024-03-23T13:11:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.227+0000 7fed4c6a3700 1 -- 172.21.15.7:0/206056770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed441a6970 con 0x7fed44199d80 2024-03-23T13:11:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.228+0000 7fed3b7fe700 1 -- 172.21.15.7:0/206056770 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c00c460 con 0x7fed44199d80 2024-03-23T13:11:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.228+0000 7fed4c6a3700 1 -- 172.21.15.7:0/206056770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441a6e70 con 0x7fed44199d80 2024-03-23T13:11:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.229+0000 7fed3b7fe700 1 -- 172.21.15.7:0/206056770 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7fed3c0156d0 con 0x7fed44199d80 2024-03-23T13:11:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.229+0000 7fed4c6a3700 1 -- 172.21.15.7:0/206056770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed44103320 con 0x7fed44199d80 2024-03-23T13:11:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.229+0000 7fed3b7fe700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed30039410 0x7fed3003b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.229+0000 7fed3b7fe700 1 -- 172.21.15.7:0/206056770 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fed3c011070 con 0x7fed44199d80 2024-03-23T13:11:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.229+0000 7fed4a43f700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed30039410 0x7fed3003b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.229+0000 7fed4a43f700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed30039410 0x7fed3003b8d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed40006010 tx=0x7fed4000b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.234+0000 7fed3b7fe700 1 -- 172.21.15.7:0/206056770 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fed3c01cb60 con 0x7fed44199d80 2024-03-23T13:11:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.458+0000 7fed4c6a3700 1 -- 172.21.15.7:0/206056770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fed44063830 con 0x7fed44199d80 2024-03-23T13:11:44.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.458+0000 7fed3b7fe700 1 -- 172.21.15.7:0/206056770 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fed3c019e80 con 0x7fed44199d80 2024-03-23T13:11:44.452 INFO:teuthology.orchestra.run.smithi007.stdout:# minimal ceph.conf for 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:44.452 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-03-23T13:11:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: mon_host = [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2024-03-23T13:11:44.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 -- 172.21.15.7:0/206056770 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed30039410 msgr2=0x7fed3003b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:44.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed30039410 0x7fed3003b8d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed40006010 tx=0x7fed4000b560 comp rx=0 tx=0).stop 2024-03-23T13:11:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 -- 172.21.15.7:0/206056770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed44199d80 msgr2=0x7fed4419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed44199d80 0x7fed4419a230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00ba50 tx=0x7fed3c00bd60 comp rx=0 tx=0).stop 2024-03-23T13:11:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 -- 172.21.15.7:0/206056770 shutdown_connections 2024-03-23T13:11:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed30039410 0x7fed3003b8d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed44199d80 0x7fed4419a230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed44101e40 0x7fed4419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 --2- 172.21.15.7:0/206056770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed4419fc00 0x7fed44199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 -- 172.21.15.7:0/206056770 >> 172.21.15.7:0/206056770 conn(0x7fed440fd6f0 msgr2=0x7fed440744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 -- 172.21.15.7:0/206056770 shutdown_connections 2024-03-23T13:11:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:44.461+0000 7fed4c6a3700 1 -- 172.21.15.7:0/206056770 wait complete. 2024-03-23T13:11:44.903 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: mon.b calling monitor election 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: Updating smithi130:/etc/ceph/ceph.conf 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: Updating smithi007:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: Updating smithi130:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: Updating smithi155:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: Reconfiguring daemon mon.a on smithi007 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: mon.b calling monitor election 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: mon.c calling monitor election 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: mon.a calling monitor election 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T13:11:44.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: monmap epoch 3 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: last_changed 2024-03-23T13:11:36.706371+0000 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: created 2024-03-23T13:09:36.205070+0000 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: min_mon_release 19 (squid) 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: election_strategy: 1 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: fsmap 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: mgrmap e13: x(active, since 61s) 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: overall HEALTH_OK 2024-03-23T13:11:44.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:44 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/206056770' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:44.937 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: mon.b calling monitor election 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: Updating smithi130:/etc/ceph/ceph.conf 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: Updating smithi007:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: Updating smithi130:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: Updating smithi155:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: Reconfiguring daemon mon.a on smithi007 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: mon.b calling monitor election 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: mon.c calling monitor election 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: mon.a calling monitor election 2024-03-23T13:11:44.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: monmap epoch 3 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: last_changed 2024-03-23T13:11:36.706371+0000 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: created 2024-03-23T13:09:36.205070+0000 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: min_mon_release 19 (squid) 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: election_strategy: 1 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: fsmap 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: mgrmap e13: x(active, since 61s) 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: overall HEALTH_OK 2024-03-23T13:11:44.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:44 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/206056770' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:45.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: mon.b calling monitor election 2024-03-23T13:11:45.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-23T13:11:45.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: Updating smithi130:/etc/ceph/ceph.conf 2024-03-23T13:11:45.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-23T13:11:45.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: Updating smithi007:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:45.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: Updating smithi130:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:45.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: Updating smithi155:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T13:11:45.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: Reconfiguring daemon mon.a on smithi007 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: mon.b calling monitor election 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: mon.c calling monitor election 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: mon.a calling monitor election 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: monmap epoch 3 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: last_changed 2024-03-23T13:11:36.706371+0000 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: created 2024-03-23T13:09:36.205070+0000 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: min_mon_release 19 (squid) 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: election_strategy: 1 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-03-23T13:11:45.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: 2: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.b 2024-03-23T13:11:45.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: fsmap 2024-03-23T13:11:45.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T13:11:45.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: mgrmap e13: x(active, since 61s) 2024-03-23T13:11:45.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: overall HEALTH_OK 2024-03-23T13:11:45.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:44 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/206056770' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:45.483 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-23T13:11:45.483 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:11:45.483 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T13:11:45.513 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:11:45.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T13:11:45.580 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:11:45.580 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T13:11:45.615 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:11:45.615 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T13:11:45.685 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:11:45.686 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T13:11:45.716 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:11:45.716 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T13:11:45.784 INFO:tasks.cephadm:Adding mgr.x on smithi007 2024-03-23T13:11:45.784 INFO:tasks.cephadm:Adding mgr.y on smithi130 2024-03-23T13:11:45.784 INFO:tasks.cephadm:Adding mgr.z on smithi155 2024-03-23T13:11:45.785 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi007=x;smithi130=y;smithi155=z' 2024-03-23T13:11:45.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:45 smithi007 ceph-mon[32184]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:45.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:45 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:45.937 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:45 smithi130 ceph-mon[36225]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:45.937 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:45 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:46.030 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:45 smithi155 ceph-mon[36157]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:46.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:45 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:11:46.060 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:11:47.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:46 smithi130 ceph-mon[36225]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:47.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:47.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:47.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:46 smithi130 ceph-mon[36225]: Reconfiguring mon.b (monmap changed)... 2024-03-23T13:11:47.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:47.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T13:11:47.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:47.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:46 smithi130 ceph-mon[36225]: Reconfiguring daemon mon.b on smithi130 2024-03-23T13:11:47.239 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:46 smithi155 ceph-mon[36157]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:47.239 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:47.239 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:47.239 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:46 smithi155 ceph-mon[36157]: Reconfiguring mon.b (monmap changed)... 2024-03-23T13:11:47.239 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:47.240 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T13:11:47.240 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:47.240 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:46 smithi155 ceph-mon[36157]: Reconfiguring daemon mon.b on smithi130 2024-03-23T13:11:47.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:46 smithi007 ceph-mon[32184]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:47.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:47.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:47.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:46 smithi007 ceph-mon[32184]: Reconfiguring mon.b (monmap changed)... 2024-03-23T13:11:47.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:47.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T13:11:47.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:47.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:46 smithi007 ceph-mon[32184]: Reconfiguring daemon mon.b on smithi130 2024-03-23T13:11:47.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.546+0000 7faf1237e700 1 -- 172.21.15.155:0/4196596571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf0c101d50 msgr2=0x7faf0c102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:47.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.546+0000 7faf1237e700 1 --2- 172.21.15.155:0/4196596571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf0c101d50 0x7faf0c102170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faefc009910 tx=0x7faefc009c20 comp rx=0 tx=0).stop 2024-03-23T13:11:47.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.546+0000 7faf1237e700 1 -- 172.21.15.155:0/4196596571 shutdown_connections 2024-03-23T13:11:47.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.546+0000 7faf1237e700 1 --2- 172.21.15.155:0/4196596571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf0c101d50 0x7faf0c102170 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:47.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.546+0000 7faf1237e700 1 -- 172.21.15.155:0/4196596571 >> 172.21.15.155:0/4196596571 conn(0x7faf0c0fd2f0 msgr2=0x7faf0c0ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:47.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.546+0000 7faf1237e700 1 -- 172.21.15.155:0/4196596571 shutdown_connections 2024-03-23T13:11:47.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.546+0000 7faf1237e700 1 -- 172.21.15.155:0/4196596571 wait complete. 2024-03-23T13:11:47.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.547+0000 7faf1237e700 1 Processor -- start 2024-03-23T13:11:47.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.547+0000 7faf1237e700 1 -- start start 2024-03-23T13:11:47.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.547+0000 7faf1237e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf0c101d50 0x7faf0c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:47.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.547+0000 7faf1237e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf0c0745f0 0x7faf0c074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:47.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.547+0000 7faf1237e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf0c074fb0 0x7faf0c1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:47.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf1237e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c1a1930 con 0x7faf0c101d50 2024-03-23T13:11:47.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf1237e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c1a1a70 con 0x7faf0c074fb0 2024-03-23T13:11:47.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf1237e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c1a1bb0 con 0x7faf0c0745f0 2024-03-23T13:11:47.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf0bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf0c101d50 0x7faf0c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:47.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf0bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf0c101d50 0x7faf0c075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.155:59020/0 (socket says 172.21.15.155:59020) 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf0bfff700 1 -- 172.21.15.155:0/1233841293 learned_addr learned my addr 172.21.15.155:0/1233841293 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf0b7fe700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf0c0745f0 0x7faf0c074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf1091b700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf0c074fb0 0x7faf0c1a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf0bfff700 1 -- 172.21.15.155:0/1233841293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf0c0745f0 msgr2=0x7faf0c074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf0bfff700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf0c0745f0 0x7faf0c074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf0bfff700 1 -- 172.21.15.155:0/1233841293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf0c074fb0 msgr2=0x7faf0c1a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf0bfff700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf0c074fb0 0x7faf0c1a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf0bfff700 1 -- 172.21.15.155:0/1233841293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc0095f0 con 0x7faf0c101d50 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.548+0000 7faf0bfff700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf0c101d50 0x7faf0c075fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faefc00bcf0 tx=0x7faefc00bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.549+0000 7faf097fa700 1 -- 172.21.15.155:0/1233841293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc01d070 con 0x7faf0c101d50 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.549+0000 7faf097fa700 1 -- 172.21.15.155:0/1233841293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faefc004e00 con 0x7faf0c101d50 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.549+0000 7faf1237e700 1 -- 172.21.15.155:0/1233841293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0c1a1e30 con 0x7faf0c101d50 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.549+0000 7faf097fa700 1 -- 172.21.15.155:0/1233841293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc022d40 con 0x7faf0c101d50 2024-03-23T13:11:47.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.549+0000 7faf1237e700 1 -- 172.21.15.155:0/1233841293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0c1a2290 con 0x7faf0c101d50 2024-03-23T13:11:47.544 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.550+0000 7faf1237e700 1 -- 172.21.15.155:0/1233841293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0c190b60 con 0x7faf0c101d50 2024-03-23T13:11:47.544 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.550+0000 7faf097fa700 1 -- 172.21.15.155:0/1233841293 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7faefc004920 con 0x7faf0c101d50 2024-03-23T13:11:47.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.550+0000 7faf097fa700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faef4039410 0x7faef403b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:47.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.550+0000 7faf097fa700 1 -- 172.21.15.155:0/1233841293 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faefc04d060 con 0x7faf0c101d50 2024-03-23T13:11:47.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.550+0000 7faf0b7fe700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faef4039410 0x7faef403b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:47.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.551+0000 7faf0b7fe700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faef4039410 0x7faef403b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf00009ad0 tx=0x7faf00009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:47.548 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.555+0000 7faf097fa700 1 -- 172.21.15.155:0/1233841293 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7faefc00e5c0 con 0x7faf0c101d50 2024-03-23T13:11:47.761 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.768+0000 7faf1237e700 1 -- 172.21.15.155:0/1233841293 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi130=y;smithi155=z", "target": ["mon-mgr", ""]}) -- 0x7faf0c02dd20 con 0x7faef4039410 2024-03-23T13:11:47.767 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.775+0000 7faf097fa700 1 -- 172.21.15.155:0/1233841293 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faf0c02dd20 con 0x7faef4039410 2024-03-23T13:11:47.768 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mgr update... 2024-03-23T13:11:47.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 -- 172.21.15.155:0/1233841293 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faef4039410 msgr2=0x7faef403b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:47.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faef4039410 0x7faef403b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf00009ad0 tx=0x7faf00009210 comp rx=0 tx=0).stop 2024-03-23T13:11:47.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 -- 172.21.15.155:0/1233841293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf0c101d50 msgr2=0x7faf0c075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:47.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf0c101d50 0x7faf0c075fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faefc00bcf0 tx=0x7faefc00bd20 comp rx=0 tx=0).stop 2024-03-23T13:11:47.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 -- 172.21.15.155:0/1233841293 shutdown_connections 2024-03-23T13:11:47.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7faef4039410 0x7faef403b8d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:47.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf0c074fb0 0x7faf0c1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:47.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf0c101d50 0x7faf0c075fa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:47.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 --2- 172.21.15.155:0/1233841293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf0c0745f0 0x7faf0c074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:47.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 -- 172.21.15.155:0/1233841293 >> 172.21.15.155:0/1233841293 conn(0x7faf0c0fd2f0 msgr2=0x7faf0c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:47.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 -- 172.21.15.155:0/1233841293 shutdown_connections 2024-03-23T13:11:47.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:11:47.777+0000 7faf1237e700 1 -- 172.21.15.155:0/1233841293 wait complete. 2024-03-23T13:11:48.371 DEBUG:teuthology.orchestra.run.smithi130:mgr.y> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.y.service 2024-03-23T13:11:48.375 DEBUG:teuthology.orchestra.run.smithi155:mgr.z> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.z.service 2024-03-23T13:11:48.378 INFO:tasks.cephadm:Deploying OSDs... 2024-03-23T13:11:48.378 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:11:48.378 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T13:11:48.397 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T13:11:48.398 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-03-23T13:11:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T13:11:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:48.455 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-23T13:11:48.455 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:48.456 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:48.456 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-23 13:10:54.699687927 +0000 2024-03-23T13:11:48.456 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-23 13:06:09.097358419 +0000 2024-03-23T13:11:48.456 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-23 13:06:09.097358419 +0000 2024-03-23T13:11:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-23 13:06:09.097358419 +0000 2024-03-23T13:11:48.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T13:11:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-23T13:11:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-23T13:11:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000121425 s, 4.2 MB/s 2024-03-23T13:11:48.525 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:48 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:48.525 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:48 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:48.525 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:48 smithi155 ceph-mon[36157]: Reconfiguring mon.c (monmap changed)... 2024-03-23T13:11:48.525 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:48 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:48.526 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:48 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T13:11:48.526 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:48 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:48.526 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:48 smithi155 ceph-mon[36157]: Reconfiguring daemon mon.c on smithi155 2024-03-23T13:11:48.526 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:48 smithi155 ceph-mon[36157]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi130=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:48.526 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:48 smithi155 ceph-mon[36157]: Saving service mgr spec with placement smithi007=x;smithi130=y;smithi155=z;count:3 2024-03-23T13:11:48.526 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:48 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:48.527 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T13:11:48.585 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-03-23T13:11:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T13:11:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:48.643 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-23T13:11:48.643 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:48.643 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:48.643 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-23 13:10:54.737687398 +0000 2024-03-23T13:11:48.643 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-23 13:06:09.435353373 +0000 2024-03-23T13:11:48.643 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-23 13:06:09.435353373 +0000 2024-03-23T13:11:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-23 13:06:09.435353373 +0000 2024-03-23T13:11:48.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T13:11:48.687 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:48 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:48.687 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:48 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:48.687 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:48 smithi130 ceph-mon[36225]: Reconfiguring mon.c (monmap changed)... 2024-03-23T13:11:48.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:48 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:48.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:48 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T13:11:48.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:48 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:48.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:48 smithi130 ceph-mon[36225]: Reconfiguring daemon mon.c on smithi155 2024-03-23T13:11:48.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:48 smithi130 ceph-mon[36225]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi130=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:48.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:48 smithi130 ceph-mon[36225]: Saving service mgr spec with placement smithi007=x;smithi130=y;smithi155=z;count:3 2024-03-23T13:11:48.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:48 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:48.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:48 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:48.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:48 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:48.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:48 smithi007 ceph-mon[32184]: Reconfiguring mon.c (monmap changed)... 2024-03-23T13:11:48.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:48 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T13:11:48.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:48 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T13:11:48.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:48 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:48.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:48 smithi007 ceph-mon[32184]: Reconfiguring daemon mon.c on smithi155 2024-03-23T13:11:48.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:48 smithi007 ceph-mon[32184]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi130=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:48.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:48 smithi007 ceph-mon[32184]: Saving service mgr spec with placement smithi007=x;smithi130=y;smithi155=z;count:3 2024-03-23T13:11:48.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:48 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:48.712 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-23T13:11:48.712 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-23T13:11:48.712 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000141338 s, 3.6 MB/s 2024-03-23T13:11:48.714 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T13:11:48.771 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-03-23T13:11:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T13:11:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:48.829 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-23T13:11:48.829 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:48.829 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:48.829 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-23 13:10:54.772686911 +0000 2024-03-23T13:11:48.829 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-23 13:06:09.754348610 +0000 2024-03-23T13:11:48.829 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-23 13:06:09.754348610 +0000 2024-03-23T13:11:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-23 13:06:09.754348610 +0000 2024-03-23T13:11:48.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T13:11:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-23T13:11:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-23T13:11:48.897 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 9.7869e-05 s, 5.2 MB/s 2024-03-23T13:11:48.898 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T13:11:48.955 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-03-23T13:11:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T13:11:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:49.012 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-23T13:11:49.012 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:49.012 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:49.012 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-23 13:10:54.806686438 +0000 2024-03-23T13:11:49.012 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-23 13:06:10.076343802 +0000 2024-03-23T13:11:49.012 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-23 13:06:10.076343802 +0000 2024-03-23T13:11:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-23 13:06:10.076343802 +0000 2024-03-23T13:11:49.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T13:11:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-23T13:11:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-23T13:11:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000104122 s, 4.9 MB/s 2024-03-23T13:11:49.080 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T13:11:49.137 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-03-23T13:11:49.138 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T13:11:49.156 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T13:11:49.157 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_1 2024-03-23T13:11:49.215 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T13:11:49.215 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:49.215 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-03-23T13:11:49.216 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:49.216 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:49.216 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-03-23 13:11:36.137434292 +0000 2024-03-23T13:11:49.216 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-03-23 13:06:23.683966827 +0000 2024-03-23T13:11:49.216 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-03-23 13:06:23.683966827 +0000 2024-03-23T13:11:49.216 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-03-23 13:06:23.683966827 +0000 2024-03-23T13:11:49.216 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T13:11:49.285 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-03-23T13:11:49.285 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-03-23T13:11:49.285 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000276428 s, 1.9 MB/s 2024-03-23T13:11:49.286 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T13:11:49.344 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_2 2024-03-23T13:11:49.402 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T13:11:49.402 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:49.402 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-23T13:11:49.402 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:49.402 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:49.402 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-03-23 13:11:36.137434292 +0000 2024-03-23T13:11:49.402 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-03-23 13:06:24.010960608 +0000 2024-03-23T13:11:49.402 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-03-23 13:06:24.010960608 +0000 2024-03-23T13:11:49.403 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-03-23 13:06:24.010960608 +0000 2024-03-23T13:11:49.403 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T13:11:49.472 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-03-23T13:11:49.472 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-03-23T13:11:49.472 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.00022417 s, 2.3 MB/s 2024-03-23T13:11:49.473 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T13:11:49.532 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_3 2024-03-23T13:11:49.590 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T13:11:49.591 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:49.591 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-23T13:11:49.591 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:49.591 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:49.591 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-03-23 13:11:36.138434274 +0000 2024-03-23T13:11:49.591 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-03-23 13:06:24.340954332 +0000 2024-03-23T13:11:49.591 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-03-23 13:06:24.340954332 +0000 2024-03-23T13:11:49.591 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-03-23 13:06:24.340954332 +0000 2024-03-23T13:11:49.591 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T13:11:49.659 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-03-23T13:11:49.660 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-03-23T13:11:49.660 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000210643 s, 2.4 MB/s 2024-03-23T13:11:49.661 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T13:11:49.719 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_4 2024-03-23T13:11:49.775 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T13:11:49.775 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:49.775 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-03-23T13:11:49.775 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:49.775 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:49.775 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-03-23 13:11:36.138434274 +0000 2024-03-23T13:11:49.775 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-03-23 13:06:24.670948056 +0000 2024-03-23T13:11:49.775 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-03-23 13:06:24.670948056 +0000 2024-03-23T13:11:49.775 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-03-23 13:06:24.670948056 +0000 2024-03-23T13:11:49.776 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T13:11:49.843 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-03-23T13:11:49.843 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-03-23T13:11:49.843 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000206615 s, 2.5 MB/s 2024-03-23T13:11:49.844 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T13:11:49.902 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-23T13:11:49.902 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T13:11:49.919 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T13:11:49.919 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-03-23T13:11:49.937 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:49 smithi130 ceph-mon[36225]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:49.937 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:49 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:49.937 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:49 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:49.937 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:49 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:49.974 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T13:11:49.974 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-23T13:11:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-23 13:11:27.323946281 +0000 2024-03-23T13:11:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-23 13:06:23.759418527 +0000 2024-03-23T13:11:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-23 13:06:23.759418527 +0000 2024-03-23T13:11:49.975 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-03-23 13:06:23.759418527 +0000 2024-03-23T13:11:49.975 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T13:11:50.030 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:49 smithi155 ceph-mon[36157]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:50.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:49 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:50.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:49 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:50.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:49 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:50.041 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-23T13:11:50.042 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-23T13:11:50.042 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000252458 s, 2.0 MB/s 2024-03-23T13:11:50.043 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T13:11:50.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:49 smithi007 ceph-mon[32184]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:50.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:49 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:50.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:49 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:50.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:49 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:50.098 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-03-23T13:11:50.153 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T13:11:50.153 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:50.154 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-23T13:11:50.154 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:50.154 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:50.154 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-23 13:11:27.323946281 +0000 2024-03-23T13:11:50.154 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-23 13:06:24.116410883 +0000 2024-03-23T13:11:50.154 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-23 13:06:24.116410883 +0000 2024-03-23T13:11:50.154 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-03-23 13:06:24.116410883 +0000 2024-03-23T13:11:50.154 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T13:11:50.219 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-23T13:11:50.219 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-23T13:11:50.220 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000176694 s, 2.9 MB/s 2024-03-23T13:11:50.220 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T13:11:50.277 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-03-23T13:11:50.332 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T13:11:50.333 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:50.333 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 698 Links: 1 2024-03-23T13:11:50.333 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:50.333 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:50.333 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-23 13:11:27.323946281 +0000 2024-03-23T13:11:50.333 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-23 13:06:24.440403946 +0000 2024-03-23T13:11:50.333 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-23 13:06:24.440403946 +0000 2024-03-23T13:11:50.333 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-03-23 13:06:24.440403946 +0000 2024-03-23T13:11:50.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T13:11:50.398 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-23T13:11:50.398 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-23T13:11:50.398 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000173615 s, 2.9 MB/s 2024-03-23T13:11:50.400 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T13:11:50.456 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-03-23T13:11:50.511 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T13:11:50.512 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T13:11:50.512 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-03-23T13:11:50.512 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:11:50.512 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T13:11:50.512 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-23 13:11:27.324946261 +0000 2024-03-23T13:11:50.512 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-23 13:06:24.798396280 +0000 2024-03-23T13:11:50.512 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-23 13:06:24.798396280 +0000 2024-03-23T13:11:50.512 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-03-23 13:06:24.798396280 +0000 2024-03-23T13:11:50.512 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T13:11:50.577 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-23T13:11:50.578 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-23T13:11:50.578 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.00018201 s, 2.8 MB/s 2024-03-23T13:11:50.579 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T13:11:50.635 INFO:tasks.cephadm:Deploying osd.0 on smithi007 with /dev/vg_nvme/lv_4... 2024-03-23T13:11:50.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T13:11:50.873 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:11:50.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:50 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:50.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:50 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:50.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:50 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:50.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:50 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T13:11:50.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:50 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T13:11:50.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:50 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T13:11:50.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:50 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:51.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:50 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:51.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:50 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:51.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:50 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:51.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:50 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T13:11:51.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:50 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T13:11:51.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:50 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T13:11:51.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:50 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:51.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:50 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:51.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:50 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:51.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:50 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:51.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:50 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T13:11:51.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:50 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T13:11:51.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:50 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T13:11:51.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:50 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:51.937 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:51 smithi155 ceph-mon[36157]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:51.938 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:51 smithi155 ceph-mon[36157]: Deploying daemon mgr.z on smithi155 2024-03-23T13:11:52.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:51 smithi007 ceph-mon[32184]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:52.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:51 smithi007 ceph-mon[32184]: Deploying daemon mgr.z on smithi155 2024-03-23T13:11:52.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:51 smithi130 ceph-mon[36225]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:52.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:51 smithi130 ceph-mon[36225]: Deploying daemon mgr.z on smithi155 2024-03-23T13:11:52.281 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:52 smithi155 systemd[1]: Starting Ceph mgr.z for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:11:52.621 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:52 smithi155 podman[38012]: 2024-03-23 13:11:52.325603478 +0000 UTC m=+0.113282723 container create 3499c355819435b9ff05ad5090cf11c468734b63cf57f560d95d4370b74c3157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-23T13:11:52.621 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:52 smithi155 podman[38012]: 2024-03-23 13:11:52.23011593 +0000 UTC m=+0.017795144 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:11:52.621 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:52 smithi155 podman[38012]: 2024-03-23 13:11:52.5027984 +0000 UTC m=+0.290477614 container init 3499c355819435b9ff05ad5090cf11c468734b63cf57f560d95d4370b74c3157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-23T13:11:52.621 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:52 smithi155 podman[38012]: 2024-03-23 13:11:52.505455584 +0000 UTC m=+0.293134786 container start 3499c355819435b9ff05ad5090cf11c468734b63cf57f560d95d4370b74c3157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:11:52.621 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:52 smithi155 ceph-mgr[38031]: -- 172.21.15.155:0/4258447081 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625ccd3a1a0 con 0x5625ccd11000 2024-03-23T13:11:52.621 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:52 smithi155 bash[38012]: 3499c355819435b9ff05ad5090cf11c468734b63cf57f560d95d4370b74c3157 2024-03-23T13:11:52.683 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T13:11:52.683 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T13:11:52.683 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-23T13:11:52.683 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-23T13:11:52.683 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00875386 s, 1.2 GB/s 2024-03-23T13:11:52.683 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-23T13:11:53.031 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:52 smithi155 systemd[1]: Started Ceph mgr.z for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:11:53.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_4 2024-03-23T13:11:53.531 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:53.057+0000 7fd42d9e5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T13:11:53.531 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:53.150+0000 7fd42d9e5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T13:11:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:11:53.937 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:53 smithi130 ceph-mon[36225]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:53.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:53.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:53.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:53.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T13:11:53.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T13:11:53.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T13:11:53.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:53.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:53 smithi130 ceph-mon[36225]: Deploying daemon mgr.y on smithi130 2024-03-23T13:11:54.010 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-mon[36157]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:54.010 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:54.010 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:54.011 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:54.011 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T13:11:54.011 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T13:11:54.011 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T13:11:54.011 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:54.011 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-mon[36157]: Deploying daemon mgr.y on smithi130 2024-03-23T13:11:54.011 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:53.785+0000 7fd42d9e5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T13:11:54.011 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:53.876+0000 7fd42d9e5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T13:11:54.011 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:54.017+0000 7fd42d9e5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T13:11:54.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:53 smithi007 ceph-mon[32184]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:54.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:54.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:54.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:54.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T13:11:54.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T13:11:54.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T13:11:54.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:54.036 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:53 smithi007 ceph-mon[32184]: Deploying daemon mgr.y on smithi130 2024-03-23T13:11:54.281 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:54.110+0000 7fd42d9e5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T13:11:54.281 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:54.192+0000 7fd42d9e5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T13:11:54.704 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:54.454+0000 7fd42d9e5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T13:11:54.704 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:54.561+0000 7fd42d9e5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T13:11:54.704 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:54.646+0000 7fd42d9e5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T13:11:55.031 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:54.743+0000 7fd42d9e5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T13:11:55.031 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:54.919+0000 7fd42d9e5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T13:11:55.187 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:55 smithi130 systemd[1]: Started Ceph mgr.y for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:11:55.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.268+0000 7f5b81b23700 1 -- 172.21.15.7:0/2103913873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b7c103430 msgr2=0x7f5b7c103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.268+0000 7f5b81b23700 1 --2- 172.21.15.7:0/2103913873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b7c103430 0x7f5b7c103890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5b700099c0 tx=0x7f5b70009cd0 comp rx=0 tx=0).stop 2024-03-23T13:11:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.269+0000 7f5b81b23700 1 -- 172.21.15.7:0/2103913873 shutdown_connections 2024-03-23T13:11:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.269+0000 7f5b81b23700 1 --2- 172.21.15.7:0/2103913873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c103dd0 0x7f5b7c10a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.269+0000 7f5b81b23700 1 --2- 172.21.15.7:0/2103913873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b7c102230 0x7f5b7c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.269+0000 7f5b81b23700 1 --2- 172.21.15.7:0/2103913873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b7c103430 0x7f5b7c103890 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.269+0000 7f5b81b23700 1 -- 172.21.15.7:0/2103913873 >> 172.21.15.7:0/2103913873 conn(0x7f5b7c0fd810 msgr2=0x7f5b7c0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:11:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.269+0000 7f5b81b23700 1 -- 172.21.15.7:0/2103913873 shutdown_connections 2024-03-23T13:11:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.269+0000 7f5b81b23700 1 -- 172.21.15.7:0/2103913873 wait complete. 2024-03-23T13:11:55.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.269+0000 7f5b81b23700 1 Processor -- start 2024-03-23T13:11:55.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b81b23700 1 -- start start 2024-03-23T13:11:55.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b81b23700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b7c102230 0x7f5b7c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b81b23700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b7c103430 0x7f5b7c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b81b23700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c103dd0 0x7f5b7c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b81b23700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c19fda0 con 0x7f5b7c103430 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b81b23700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c19fee0 con 0x7f5b7c102230 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b81b23700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c1a0070 con 0x7f5b7c103dd0 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b7c103430 0x7f5b7c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b7c103430 0x7f5b7c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52042/0 (socket says 172.21.15.7:52042) 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7affd700 1 -- 172.21.15.7:0/1308953294 learned_addr learned my addr 172.21.15.7:0/1308953294 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7b7fe700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b7c102230 0x7f5b7c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7bfff700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c103dd0 0x7f5b7c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7affd700 1 -- 172.21.15.7:0/1308953294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c103dd0 msgr2=0x7f5b7c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7affd700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c103dd0 0x7f5b7c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7affd700 1 -- 172.21.15.7:0/1308953294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b7c102230 msgr2=0x7f5b7c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7affd700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b7c102230 0x7f5b7c198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7affd700 1 -- 172.21.15.7:0/1308953294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b700095f0 con 0x7f5b7c103430 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.270+0000 7f5b7affd700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b7c103430 0x7f5b7c198750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009f50 tx=0x7f5b70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.271+0000 7f5b78ff9700 1 -- 172.21.15.7:0/1308953294 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7001d070 con 0x7f5b7c103430 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.271+0000 7f5b78ff9700 1 -- 172.21.15.7:0/1308953294 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b70004050 con 0x7f5b7c103430 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.271+0000 7f5b81b23700 1 -- 172.21.15.7:0/1308953294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7c1a02f0 con 0x7f5b7c103430 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.271+0000 7f5b78ff9700 1 -- 172.21.15.7:0/1308953294 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70022e60 con 0x7f5b7c103430 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.271+0000 7f5b81b23700 1 -- 172.21.15.7:0/1308953294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7c1a0760 con 0x7f5b7c103430 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.271+0000 7f5b78ff9700 1 -- 172.21.15.7:0/1308953294 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 45983+0+0 (secure 0 0 0) 0x7f5b7000f460 con 0x7f5b7c103430 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.272+0000 7f5b81b23700 1 -- 172.21.15.7:0/1308953294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7c1096c0 con 0x7f5b7c103430 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.272+0000 7f5b78ff9700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f5b640391c0 0x7f5b6403b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.272+0000 7f5b78ff9700 1 -- 172.21.15.7:0/1308953294 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5b7004ccf0 con 0x7f5b7c103430 2024-03-23T13:11:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.272+0000 7f5b7b7fe700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f5b640391c0 0x7f5b6403b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:11:55.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.272+0000 7f5b7b7fe700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f5b640391c0 0x7f5b6403b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c103290 tx=0x7f5b6c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:11:55.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.277+0000 7f5b78ff9700 1 -- 172.21.15.7:0/1308953294 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f5b70016990 con 0x7f5b7c103430 2024-03-23T13:11:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:55.505+0000 7f5b81b23700 1 -- 172.21.15.7:0/1308953294 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5b7c02db60 con 0x7f5b640391c0 2024-03-23T13:11:55.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:55 smithi007 ceph-mon[32184]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:55.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:55.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:55.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:55.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:55.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:55.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:11:55.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:11:55.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:55.937 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:55.585+0000 7f05c7348200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T13:11:55.938 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:55.682+0000 7f05c7348200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T13:11:55.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-mon[36225]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:55.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:55.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:55.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:55.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:55.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:55.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:11:55.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:11:55.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:56.031 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:55.923+0000 7fd42d9e5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T13:11:56.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:55 smithi155 ceph-mon[36157]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:56.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:56.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:56.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:56.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:56.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:11:56.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:11:56.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:11:56.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:56.351 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:56 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:56.068+0000 7fd42d9e5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T13:11:56.596 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:56 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:56.366+0000 7f05c7348200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T13:11:56.596 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:56 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:56.460+0000 7f05c7348200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T13:11:56.702 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:56 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:56.413+0000 7fd42d9e5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T13:11:56.702 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:56 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:56.503+0000 7fd42d9e5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T13:11:56.702 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:56 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:56.709+0000 7fd42d9e5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T13:11:56.937 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:56 smithi130 ceph-mon[36225]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:56.938 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:56 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:56.603+0000 7f05c7348200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T13:11:56.938 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:56 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:56.704+0000 7f05c7348200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T13:11:56.938 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:56 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:56.791+0000 7f05c7348200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T13:11:57.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:56 smithi155 ceph-mon[36157]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:57.031 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:56 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:56.893+0000 7fd42d9e5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T13:11:57.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:56 smithi007 ceph-mon[32184]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:11:57.341 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:57.057+0000 7f05c7348200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T13:11:57.341 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:57.167+0000 7f05c7348200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T13:11:57.341 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:57.252+0000 7f05c7348200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T13:11:57.341 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:57.347+0000 7f05c7348200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T13:11:57.426 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:57.174+0000 7fd42d9e5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T13:11:57.426 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:57.432+0000 7fd42d9e5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T13:11:57.690 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:57.522+0000 7f05c7348200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T13:11:57.733 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:57.665+0000 7fd42d9e5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T13:11:58.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:58.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:58.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:58.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T13:11:58.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T13:11:58.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:57 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:58.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:58.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T13:11:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T13:11:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:57 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:58.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T13:11:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:57 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:11:58.736 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:58 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:58.489+0000 7f05c7348200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T13:11:58.737 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:58 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:58.619+0000 7f05c7348200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T13:11:58.737 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:58 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:58.459+0000 7fd42d9e5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T13:11:59.016 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:58 smithi155 ceph-mon[36157]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T13:11:59.016 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:58 smithi155 ceph-mon[36157]: Reconfiguring daemon mgr.x on smithi007 2024-03-23T13:11:59.017 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:58 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3217070036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0da10b6e-bba6-44c4-ae11-5378f134eb22"}]: dispatch 2024-03-23T13:11:59.017 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:58 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3217070036' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0da10b6e-bba6-44c4-ae11-5378f134eb22"}]': finished 2024-03-23T13:11:59.017 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:58 smithi155 ceph-mon[36157]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T13:11:59.017 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:58 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:11:59.036 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:58 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:58.954+0000 7f05c7348200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T13:11:59.037 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:59.043+0000 7f05c7348200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T13:11:59.037 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:58 smithi130 ceph-mon[36225]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T13:11:59.037 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:58 smithi130 ceph-mon[36225]: Reconfiguring daemon mgr.x on smithi007 2024-03-23T13:11:59.037 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:58 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3217070036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0da10b6e-bba6-44c4-ae11-5378f134eb22"}]: dispatch 2024-03-23T13:11:59.037 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:58 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3217070036' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0da10b6e-bba6-44c4-ae11-5378f134eb22"}]': finished 2024-03-23T13:11:59.037 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:58 smithi130 ceph-mon[36225]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T13:11:59.037 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:58 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:11:59.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:58 smithi007 ceph-mon[32184]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T13:11:59.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:58 smithi007 ceph-mon[32184]: Reconfiguring daemon mgr.x on smithi007 2024-03-23T13:11:59.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:58 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3217070036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0da10b6e-bba6-44c4-ae11-5378f134eb22"}]: dispatch 2024-03-23T13:11:59.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:58 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3217070036' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0da10b6e-bba6-44c4-ae11-5378f134eb22"}]': finished 2024-03-23T13:11:59.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:58 smithi007 ceph-mon[32184]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T13:11:59.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:11:59.281 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:59 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:59.023+0000 7fd42d9e5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T13:11:59.281 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:59 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:59.119+0000 7fd42d9e5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T13:11:59.417 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:59.238+0000 7f05c7348200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T13:11:59.417 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:59.423+0000 7f05c7348200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T13:11:59.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:11:59.748+0000 7f5b78ff9700 1 -- 172.21.15.7:0/1308953294 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 14) ==== 91341+0+0 (secure 0 0 0) 0x7f5b7002c440 con 0x7f5b7c103430 2024-03-23T13:11:59.746 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:11:59 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z[38027]: 2024-03-23T13:11:59.331+0000 7fd42d9e5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T13:11:59.961 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:59.706+0000 7f05c7348200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T13:11:59.961 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:11:59.968+0000 7f05c7348200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T13:11:59.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-mon[36225]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:11:59.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1978317849' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:11:59.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-mon[36225]: Standby manager daemon z started 2024-03-23T13:11:59.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-mon[36225]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T13:11:59.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-mon[36225]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T13:11:59.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-mon[36225]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T13:11:59.962 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:11:59 smithi130 ceph-mon[36225]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T13:12:00.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:59 smithi155 ceph-mon[36157]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:00.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:59 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1978317849' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:12:00.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:59 smithi155 ceph-mon[36157]: Standby manager daemon z started 2024-03-23T13:12:00.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:59 smithi155 ceph-mon[36157]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T13:12:00.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:59 smithi155 ceph-mon[36157]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T13:12:00.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:59 smithi155 ceph-mon[36157]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T13:12:00.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:11:59 smithi155 ceph-mon[36157]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T13:12:00.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:59 smithi007 ceph-mon[32184]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:00.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:59 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1978317849' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:12:00.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:59 smithi007 ceph-mon[32184]: Standby manager daemon z started 2024-03-23T13:12:00.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:59 smithi007 ceph-mon[32184]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T13:12:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:59 smithi007 ceph-mon[32184]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T13:12:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:59 smithi007 ceph-mon[32184]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T13:12:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:11:59 smithi007 ceph-mon[32184]: from='mgr.? 172.21.15.155:0/3007049428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T13:12:00.437 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:12:00 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:12:00.202+0000 7f05c7348200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T13:12:01.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:00 smithi155 ceph-mon[36157]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-23T13:12:01.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:00 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T13:12:01.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:00 smithi007 ceph-mon[32184]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-23T13:12:01.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:00 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T13:12:01.187 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:12:00 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:12:00.998+0000 7f05c7348200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T13:12:01.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:00 smithi130 ceph-mon[36225]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-23T13:12:01.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:00 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T13:12:01.866 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:01 smithi130 ceph-mon[36225]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:01.867 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:12:01 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:12:01.560+0000 7f05c7348200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T13:12:01.867 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:12:01 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:12:01.658+0000 7f05c7348200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T13:12:01.867 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:12:01 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y[37606]: 2024-03-23T13:12:01.873+0000 7f05c7348200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T13:12:02.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:01 smithi155 ceph-mon[36157]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:02.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:01 smithi007 ceph-mon[32184]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:02.768+0000 7f5b78ff9700 1 -- 172.21.15.7:0/1308953294 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 15) ==== 136699+0+0 (secure 0 0 0) 0x7f5b70016bc0 con 0x7f5b7c103430 2024-03-23T13:12:02.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:02 smithi007 ceph-mon[32184]: Standby manager daemon y started 2024-03-23T13:12:02.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:02 smithi007 ceph-mon[32184]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T13:12:02.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:02 smithi007 ceph-mon[32184]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T13:12:02.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:02 smithi007 ceph-mon[32184]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T13:12:02.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:02 smithi007 ceph-mon[32184]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T13:12:03.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:02 smithi155 ceph-mon[36157]: Standby manager daemon y started 2024-03-23T13:12:03.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:02 smithi155 ceph-mon[36157]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T13:12:03.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:02 smithi155 ceph-mon[36157]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T13:12:03.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:02 smithi155 ceph-mon[36157]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T13:12:03.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:02 smithi155 ceph-mon[36157]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T13:12:03.089 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:02 smithi130 ceph-mon[36225]: Standby manager daemon y started 2024-03-23T13:12:03.089 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:02 smithi130 ceph-mon[36225]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T13:12:03.089 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:02 smithi130 ceph-mon[36225]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T13:12:03.089 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:02 smithi130 ceph-mon[36225]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T13:12:03.089 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:02 smithi130 ceph-mon[36225]: from='mgr.? 172.21.15.130:0/3266370155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T13:12:04.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:03 smithi155 ceph-mon[36157]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:04.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:03 smithi155 ceph-mon[36157]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-23T13:12:04.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T13:12:04.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:04.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:04.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:04.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:04.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:04.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:04.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:03 smithi007 ceph-mon[32184]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:04.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:03 smithi007 ceph-mon[32184]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-23T13:12:04.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T13:12:04.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:04.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:04.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:04.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:04.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:04.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:04.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:03 smithi130 ceph-mon[36225]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:04.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:03 smithi130 ceph-mon[36225]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-23T13:12:04.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T13:12:04.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:04.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:04.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:04.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:04.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:04.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:05.893 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:05 smithi007 ceph-mon[32184]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:06.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:05 smithi155 ceph-mon[36157]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:06.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:05 smithi130 ceph-mon[36225]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:07.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T13:12:07.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:07.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T13:12:07.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:07.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T13:12:07.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:08.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:07 smithi155 ceph-mon[36157]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:08.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:07 smithi155 ceph-mon[36157]: Deploying daemon osd.0 on smithi007 2024-03-23T13:12:08.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:07 smithi007 ceph-mon[32184]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:08.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:07 smithi007 ceph-mon[32184]: Deploying daemon osd.0 on smithi007 2024-03-23T13:12:08.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:07 smithi130 ceph-mon[36225]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:08.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:07 smithi130 ceph-mon[36225]: Deploying daemon osd.0 on smithi007 2024-03-23T13:12:09.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:09 smithi007 ceph-mon[32184]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:10.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:09 smithi155 ceph-mon[36157]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:10.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:09 smithi130 ceph-mon[36225]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:12.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:11 smithi155 ceph-mon[36157]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:12.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:11 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:12.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:11 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:12.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:11 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:12.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:11 smithi007 ceph-mon[32184]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:12.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:11 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:12.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:11 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:12.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:11 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:12.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:11 smithi130 ceph-mon[36225]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:12.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:11 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:12.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:11 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:12.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:11 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:14.031 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:13 smithi155 ceph-mon[36157]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:14.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:13 smithi007 ceph-mon[32184]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:14.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:13 smithi130 ceph-mon[36225]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:15.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:15 smithi007 ceph-mon[32184]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:15.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:15 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:15.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:15 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:15.171 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:15 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:15.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:15 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:15.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:15 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:15.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.261+0000 7f5b78ff9700 1 -- 172.21.15.7:0/1308953294 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b7c02db60 con 0x7f5b640391c0 2024-03-23T13:12:15.255 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 0 on host 'smithi007' 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.263+0000 7f5b81b23700 1 -- 172.21.15.7:0/1308953294 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f5b640391c0 msgr2=0x7f5b6403b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.263+0000 7f5b81b23700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f5b640391c0 0x7f5b6403b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c103290 tx=0x7f5b6c007400 comp rx=0 tx=0).stop 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.263+0000 7f5b81b23700 1 -- 172.21.15.7:0/1308953294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b7c103430 msgr2=0x7f5b7c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.263+0000 7f5b81b23700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b7c103430 0x7f5b7c198750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009f50 tx=0x7f5b70004730 comp rx=0 tx=0).stop 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.263+0000 7f5b81b23700 1 -- 172.21.15.7:0/1308953294 shutdown_connections 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.263+0000 7f5b81b23700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f5b640391c0 0x7f5b6403b680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.263+0000 7f5b81b23700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c103dd0 0x7f5b7c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.263+0000 7f5b81b23700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b7c102230 0x7f5b7c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.263+0000 7f5b81b23700 1 --2- 172.21.15.7:0/1308953294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b7c103430 0x7f5b7c198750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.263+0000 7f5b81b23700 1 -- 172.21.15.7:0/1308953294 >> 172.21.15.7:0/1308953294 conn(0x7f5b7c0fd810 msgr2=0x7f5b7c106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.264+0000 7f5b81b23700 1 -- 172.21.15.7:0/1308953294 shutdown_connections 2024-03-23T13:12:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:12:15.264+0000 7f5b81b23700 1 -- 172.21.15.7:0/1308953294 wait complete. 2024-03-23T13:12:15.438 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:15 smithi130 ceph-mon[36225]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:15.438 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:15 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:15.438 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:15 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:15.438 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:15 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:15.438 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:15 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:15.438 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:15 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:15.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:15 smithi155 ceph-mon[36157]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:15.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:15 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:15.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:15 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:15.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:15 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:15.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:15 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:15.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:15 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:15.884 DEBUG:teuthology.orchestra.run.smithi007:osd.0> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.0.service 2024-03-23T13:12:15.886 INFO:tasks.cephadm:Deploying osd.1 on smithi130 with /dev/vg_nvme/lv_4... 2024-03-23T13:12:15.886 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T13:12:16.131 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:12:16.428 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:16 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:16.428 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:16 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:16.428 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:16 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:16.428 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:16 smithi007 ceph-mon[32184]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T13:12:16.475 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:16 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:16.475 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:16 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:16.475 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:16 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:16.475 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:16 smithi130 ceph-mon[36225]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T13:12:16.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:16 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:16.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:16 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:16.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:16 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:16.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:16 smithi155 ceph-mon[36157]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T13:12:17.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:17 smithi007 ceph-mon[32184]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:17.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:17 smithi007 ceph-mon[32184]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T13:12:17.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:17 smithi007 ceph-mon[32184]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T13:12:17.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:17 smithi007 ceph-mon[32184]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:12:17.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:17.396 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:12:17 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0[47251]: 2024-03-23T13:12:17.228+0000 7f8e72f34700 -1 osd.0 0 waiting for initial osdmap 2024-03-23T13:12:17.396 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:12:17 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0[47251]: 2024-03-23T13:12:17.231+0000 7f8e6ed27700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:12:17.494 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:17 smithi130 ceph-mon[36225]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:17.494 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:17 smithi130 ceph-mon[36225]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T13:12:17.495 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:17 smithi130 ceph-mon[36225]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T13:12:17.495 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:17 smithi130 ceph-mon[36225]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:12:17.495 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:17.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:17 smithi155 ceph-mon[36157]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:17.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:17 smithi155 ceph-mon[36157]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T13:12:17.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:17 smithi155 ceph-mon[36157]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T13:12:17.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:17 smithi155 ceph-mon[36157]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:12:17.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:18.016 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T13:12:18.016 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T13:12:18.016 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-03-23T13:12:18.016 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-03-23T13:12:18.016 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0136266 s, 770 MB/s 2024-03-23T13:12:18.016 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 2024-03-23T13:12:18.016 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-03-23T13:12:18.481 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:18 smithi130 ceph-mon[36225]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:12:18.481 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:18 smithi130 ceph-mon[36225]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T13:12:18.481 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:18 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:18.481 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:18 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:18.521 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:18 smithi007 ceph-mon[32184]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:12:18.522 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:18 smithi007 ceph-mon[32184]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T13:12:18.522 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:18 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:18.522 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:18 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:18.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:18 smithi155 ceph-mon[36157]: from='osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:12:18.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:18 smithi155 ceph-mon[36157]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T13:12:18.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:18 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:18.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:18 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:18.634 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi130:vg_nvme/lv_4 2024-03-23T13:12:18.880 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] boot 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:12:19.279 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.280 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:19.280 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:19.280 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:12:19.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:12:19.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:19.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:19.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] boot 2024-03-23T13:12:19.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T13:12:19.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:19.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:12:19.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:19.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:19.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:12:19.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:12:19.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T13:12:19.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:19.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: osd.0 [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] boot 2024-03-23T13:12:19.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T13:12:19.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T13:12:19.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:12:19.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:19.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:19.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:19.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:20.326 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:20 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi007 2024-03-23T13:12:20.327 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:20 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi007 to 17093M 2024-03-23T13:12:20.327 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:20 smithi130 ceph-mon[36225]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T13:12:20.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.435+0000 7f529a139700 1 -- 172.21.15.130:0/4206422306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52941034d0 msgr2=0x7f5294103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:12:20.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.435+0000 7f529a139700 1 --2- 172.21.15.130:0/4206422306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52941034d0 0x7f5294103930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5284009990 tx=0x7f5284009ca0 comp rx=0 tx=0).stop 2024-03-23T13:12:20.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.435+0000 7f529a139700 1 -- 172.21.15.130:0/4206422306 shutdown_connections 2024-03-23T13:12:20.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.435+0000 7f529a139700 1 --2- 172.21.15.130:0/4206422306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5294103e70 0x7f529410a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:20.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.435+0000 7f529a139700 1 --2- 172.21.15.130:0/4206422306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52941004f0 0x7f5294100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:20.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.435+0000 7f529a139700 1 --2- 172.21.15.130:0/4206422306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52941034d0 0x7f5294103930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:20.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.435+0000 7f529a139700 1 -- 172.21.15.130:0/4206422306 >> 172.21.15.130:0/4206422306 conn(0x7f52940fbe70 msgr2=0x7f52940fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:12:20.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.435+0000 7f529a139700 1 -- 172.21.15.130:0/4206422306 shutdown_connections 2024-03-23T13:12:20.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.435+0000 7f529a139700 1 -- 172.21.15.130:0/4206422306 wait complete. 2024-03-23T13:12:20.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f529a139700 1 Processor -- start 2024-03-23T13:12:20.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f529a139700 1 -- start start 2024-03-23T13:12:20.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f529a139700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52941004f0 0x7f529419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:12:20.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f529a139700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52941034d0 0x7f529419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:12:20.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f529a139700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5294103e70 0x7f52941a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:12:20.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f529a139700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52941a1fd0 con 0x7f52941034d0 2024-03-23T13:12:20.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f529a139700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f52941a2110 con 0x7f52941004f0 2024-03-23T13:12:20.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f529a139700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52941a22a0 con 0x7f5294103e70 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f52937fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52941004f0 0x7f529419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f52937fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52941004f0 0x7f529419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:41818/0 (socket says 172.21.15.130:41818) 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f52937fe700 1 -- 172.21.15.130:0/2350224200 learned_addr learned my addr 172.21.15.130:0/2350224200 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f5292ffd700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52941034d0 0x7f529419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f52937fe700 1 -- 172.21.15.130:0/2350224200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5294103e70 msgr2=0x7f52941a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f52937fe700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5294103e70 0x7f52941a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f52937fe700 1 -- 172.21.15.130:0/2350224200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52941034d0 msgr2=0x7f529419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f52937fe700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52941034d0 0x7f529419a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.436+0000 7f52937fe700 1 -- 172.21.15.130:0/2350224200 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52840095f0 con 0x7f52941004f0 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.437+0000 7f52937fe700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52941004f0 0x7f529419a440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f528800d980 tx=0x7f528800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.437+0000 7f5290ff9700 1 -- 172.21.15.130:0/2350224200 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528800de60 con 0x7f52941004f0 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.437+0000 7f5290ff9700 1 -- 172.21.15.130:0/2350224200 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5288004500 con 0x7f52941004f0 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.437+0000 7f529a139700 1 -- 172.21.15.130:0/2350224200 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52941a2580 con 0x7f52941004f0 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.437+0000 7f5290ff9700 1 -- 172.21.15.130:0/2350224200 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288003ce0 con 0x7f52941004f0 2024-03-23T13:12:20.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.437+0000 7f529a139700 1 -- 172.21.15.130:0/2350224200 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52941a2ad0 con 0x7f52941004f0 2024-03-23T13:12:20.433 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.438+0000 7f529a139700 1 -- 172.21.15.130:0/2350224200 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5294068ec0 con 0x7f52941004f0 2024-03-23T13:12:20.433 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.440+0000 7f5290ff9700 1 -- 172.21.15.130:0/2350224200 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 15) ==== 136699+0+0 (secure 0 0 0) 0x7f5288009940 con 0x7f52941004f0 2024-03-23T13:12:20.434 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.440+0000 7f5290ff9700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f527c0a2140 0x7f527c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:12:20.434 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.440+0000 7f5290ff9700 1 -- 172.21.15.130:0/2350224200 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f52880cc1e0 con 0x7f52941004f0 2024-03-23T13:12:20.434 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.440+0000 7f5292ffd700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f527c0a2140 0x7f527c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:12:20.435 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.441+0000 7f5292ffd700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f527c0a2140 0x7f527c0a4600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f52840095c0 tx=0x7f5284009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:12:20.437 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.444+0000 7f5290ff9700 1 -- 172.21.15.130:0/2350224200 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f5288096ca0 con 0x7f52941004f0 2024-03-23T13:12:20.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:20 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi007 2024-03-23T13:12:20.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:20 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi007 to 17093M 2024-03-23T13:12:20.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:20 smithi155 ceph-mon[36157]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T13:12:20.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:20 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi007 2024-03-23T13:12:20.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:20 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi007 to 17093M 2024-03-23T13:12:20.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:20 smithi007 ceph-mon[32184]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T13:12:20.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:20.644+0000 7f529a139700 1 -- 172.21.15.130:0/2350224200 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f52941029e0 con 0x7f527c0a2140 2024-03-23T13:12:21.529 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:21 smithi130 ceph-mon[36225]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:21.529 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:21 smithi130 ceph-mon[36225]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:12:21.529 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:12:21.530 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:12:21.530 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:21.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:21 smithi155 ceph-mon[36157]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:21.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:21 smithi155 ceph-mon[36157]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:12:21.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:12:21.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:12:21.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:21.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:21 smithi007 ceph-mon[32184]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:21.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:21 smithi007 ceph-mon[32184]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:12:21.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:12:21.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:12:21.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:23.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:23 smithi155 ceph-mon[36157]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:23.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:23 smithi155 ceph-mon[36157]: from='client.? 172.21.15.130:0/2404648684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52bc95ae-72ec-43dd-b7b7-7a88b6593c0f"}]: dispatch 2024-03-23T13:12:23.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:23 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52bc95ae-72ec-43dd-b7b7-7a88b6593c0f"}]: dispatch 2024-03-23T13:12:23.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:23 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52bc95ae-72ec-43dd-b7b7-7a88b6593c0f"}]': finished 2024-03-23T13:12:23.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:23 smithi155 ceph-mon[36157]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T13:12:23.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:23 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:23.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:23 smithi007 ceph-mon[32184]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:23.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:23 smithi007 ceph-mon[32184]: from='client.? 172.21.15.130:0/2404648684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52bc95ae-72ec-43dd-b7b7-7a88b6593c0f"}]: dispatch 2024-03-23T13:12:23.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:23 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52bc95ae-72ec-43dd-b7b7-7a88b6593c0f"}]: dispatch 2024-03-23T13:12:23.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:23 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52bc95ae-72ec-43dd-b7b7-7a88b6593c0f"}]': finished 2024-03-23T13:12:23.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:23 smithi007 ceph-mon[32184]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T13:12:23.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:23.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:23 smithi130 ceph-mon[36225]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:23.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:23 smithi130 ceph-mon[36225]: from='client.? 172.21.15.130:0/2404648684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52bc95ae-72ec-43dd-b7b7-7a88b6593c0f"}]: dispatch 2024-03-23T13:12:23.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:23 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52bc95ae-72ec-43dd-b7b7-7a88b6593c0f"}]: dispatch 2024-03-23T13:12:23.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:23 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52bc95ae-72ec-43dd-b7b7-7a88b6593c0f"}]': finished 2024-03-23T13:12:23.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:23 smithi130 ceph-mon[36225]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T13:12:23.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:23 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:24.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:24 smithi155 ceph-mon[36157]: from='client.? 172.21.15.130:0/2816322286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:12:24.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:24 smithi007 ceph-mon[32184]: from='client.? 172.21.15.130:0/2816322286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:12:24.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:24 smithi130 ceph-mon[36225]: from='client.? 172.21.15.130:0/2816322286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:12:25.531 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:25 smithi155 ceph-mon[36157]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:25.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:25 smithi007 ceph-mon[32184]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:25.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:25 smithi130 ceph-mon[36225]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:27.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:27 smithi155 ceph-mon[36157]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:27.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:27 smithi007 ceph-mon[32184]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:27.688 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:27 smithi130 ceph-mon[36225]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:29.355 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:29 smithi130 ceph-mon[36225]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:29.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:29 smithi155 ceph-mon[36157]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:29.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:29 smithi007 ceph-mon[32184]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:30.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:30 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T13:12:30.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:30 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:30.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:30 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T13:12:30.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:30 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:30.627 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:30 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T13:12:30.627 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:30 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:31.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:31 smithi155 ceph-mon[36157]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:31.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:31 smithi155 ceph-mon[36157]: Deploying daemon osd.1 on smithi130 2024-03-23T13:12:31.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:31 smithi007 ceph-mon[32184]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:31.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:31 smithi007 ceph-mon[32184]: Deploying daemon osd.1 on smithi130 2024-03-23T13:12:31.575 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:31 smithi130 ceph-mon[36225]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:31.575 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:31 smithi130 ceph-mon[36225]: Deploying daemon osd.1 on smithi130 2024-03-23T13:12:33.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:33 smithi155 ceph-mon[36157]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:33.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:33 smithi007 ceph-mon[32184]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:33.584 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:33 smithi130 ceph-mon[36225]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:35.394 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:35 smithi130 ceph-mon[36225]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:35.394 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:35.394 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:35.394 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:35.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:35 smithi155 ceph-mon[36157]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:35.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:35.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:35.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:35.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:35 smithi007 ceph-mon[32184]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:35.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:35.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:35.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:37.386 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:37 smithi130 ceph-mon[36225]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:37.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:37 smithi155 ceph-mon[36157]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:37.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:37 smithi007 ceph-mon[32184]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:38.505 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.511+0000 7f5290ff9700 1 -- 172.21.15.130:0/2350224200 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f52941029e0 con 0x7f527c0a2140 2024-03-23T13:12:38.505 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 1 on host 'smithi130' 2024-03-23T13:12:38.507 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.513+0000 7f529a139700 1 -- 172.21.15.130:0/2350224200 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f527c0a2140 msgr2=0x7f527c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.513+0000 7f529a139700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f527c0a2140 0x7f527c0a4600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f52840095c0 tx=0x7f5284009510 comp rx=0 tx=0).stop 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.513+0000 7f529a139700 1 -- 172.21.15.130:0/2350224200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52941004f0 msgr2=0x7f529419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.513+0000 7f529a139700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52941004f0 0x7f529419a440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f528800d980 tx=0x7f528800dd40 comp rx=0 tx=0).stop 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.513+0000 7f529a139700 1 -- 172.21.15.130:0/2350224200 shutdown_connections 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.513+0000 7f529a139700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f527c0a2140 0x7f527c0a4600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.513+0000 7f529a139700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5294103e70 0x7f52941a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.513+0000 7f529a139700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52941004f0 0x7f529419a440 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.513+0000 7f529a139700 1 --2- 172.21.15.130:0/2350224200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52941034d0 0x7f529419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.513+0000 7f529a139700 1 -- 172.21.15.130:0/2350224200 >> 172.21.15.130:0/2350224200 conn(0x7f52940fbe70 msgr2=0x7f52941012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.514+0000 7f529a139700 1 -- 172.21.15.130:0/2350224200 shutdown_connections 2024-03-23T13:12:38.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:12:38.514+0000 7f529a139700 1 -- 172.21.15.130:0/2350224200 wait complete. 2024-03-23T13:12:39.041 DEBUG:teuthology.orchestra.run.smithi130:osd.1> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.1.service 2024-03-23T13:12:39.044 INFO:tasks.cephadm:Deploying osd.2 on smithi155 with /dev/vg_nvme/lv_4... 2024-03-23T13:12:39.044 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T13:12:39.289 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:12:39.662 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:39 smithi155 ceph-mon[36157]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:39.663 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.663 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.663 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.663 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.663 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:39 smithi155 ceph-mon[36157]: from='osd.1 [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T13:12:39.663 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:39 smithi155 ceph-mon[36157]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T13:12:39.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:39 smithi130 ceph-mon[36225]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:39.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:39 smithi130 ceph-mon[36225]: from='osd.1 [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T13:12:39.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:39 smithi130 ceph-mon[36225]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T13:12:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:39 smithi007 ceph-mon[32184]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:39 smithi007 ceph-mon[32184]: from='osd.1 [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T13:12:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:39 smithi007 ceph-mon[32184]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T13:12:40.938 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:40 smithi155 ceph-mon[36157]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T13:12:40.938 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:40 smithi155 ceph-mon[36157]: from='osd.1 [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:12:40.938 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:40 smithi155 ceph-mon[36157]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T13:12:40.938 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:40 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:40.938 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:40 smithi155 ceph-mon[36157]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:12:41.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:40 smithi007 ceph-mon[32184]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T13:12:41.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:40 smithi007 ceph-mon[32184]: from='osd.1 [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:12:41.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:40 smithi007 ceph-mon[32184]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T13:12:41.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:40 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:41.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:40 smithi007 ceph-mon[32184]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:12:41.186 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:40 smithi130 ceph-mon[36225]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T13:12:41.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:40 smithi130 ceph-mon[36225]: from='osd.1 [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:12:41.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:40 smithi130 ceph-mon[36225]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T13:12:41.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:40 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:41.187 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:40 smithi130 ceph-mon[36225]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:12:41.307 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T13:12:41.308 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T13:12:41.308 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-23T13:12:41.308 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-23T13:12:41.308 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122159 s, 858 MB/s 2024-03-23T13:12:41.308 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-23T13:12:41.439 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:12:41 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1[40822]: 2024-03-23T13:12:41.191+0000 7fc7eb2e7700 -1 osd.1 0 waiting for initial osdmap 2024-03-23T13:12:41.439 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:12:41 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1[40822]: 2024-03-23T13:12:41.195+0000 7fc7e30d2700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:12:41.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:41 smithi155 ceph-mon[36157]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:41.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:41 smithi155 ceph-mon[36157]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:12:41.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:41 smithi155 ceph-mon[36157]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T13:12:41.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:41.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:41.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:41 smithi155 ceph-mon[36157]: from='osd.1 ' entity='osd.1' 2024-03-23T13:12:41.936 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-03-23T13:12:41.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:41 smithi130 ceph-mon[36225]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:41.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:41 smithi130 ceph-mon[36225]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:12:41.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:41 smithi130 ceph-mon[36225]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T13:12:41.938 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:41 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:41.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:41 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:41.939 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:41 smithi130 ceph-mon[36225]: from='osd.1 ' entity='osd.1' 2024-03-23T13:12:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:41 smithi007 ceph-mon[32184]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:41 smithi007 ceph-mon[32184]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:12:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:41 smithi007 ceph-mon[32184]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T13:12:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:41 smithi007 ceph-mon[32184]: from='osd.1 ' entity='osd.1' 2024-03-23T13:12:42.176 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:12:42.989 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:12:42.989 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:12:42.989 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:42.989 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: osd.1 [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] boot 2024-03-23T13:12:42.989 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:42.990 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:42.991 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:12:43.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:12:43.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:43.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: osd.1 [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] boot 2024-03-23T13:12:43.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T13:12:43.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:12:43.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:12:43.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:43.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: osd.1 [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] boot 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:43.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:12:43.191 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:12:43.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 -- 172.21.15.155:0/1304329250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67f0102d90 msgr2=0x7f67f0105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:12:43.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 --2- 172.21.15.155:0/1304329250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67f0102d90 0x7f67f0105180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009990 tx=0x7f67e0009ca0 comp rx=0 tx=0).stop 2024-03-23T13:12:43.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 -- 172.21.15.155:0/1304329250 shutdown_connections 2024-03-23T13:12:43.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 --2- 172.21.15.155:0/1304329250 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67f0107ff0 0x7f67f010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:43.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 --2- 172.21.15.155:0/1304329250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67f0102d90 0x7f67f0105180 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:43.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 --2- 172.21.15.155:0/1304329250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f01056c0 0x7f67f0107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:43.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 -- 172.21.15.155:0/1304329250 >> 172.21.15.155:0/1304329250 conn(0x7f67f00fc760 msgr2=0x7f67f00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 -- 172.21.15.155:0/1304329250 shutdown_connections 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 -- 172.21.15.155:0/1304329250 wait complete. 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 Processor -- start 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.776+0000 7f67f4864700 1 -- start start 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67f4864700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f0102d90 0x7f67f0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67f4864700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67f01056c0 0x7f67f0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67f4864700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67f0107ff0 0x7f67f019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67f4864700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67f019fdc0 con 0x7f67f0102d90 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67f4864700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f67f019ff00 con 0x7f67f0107ff0 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67f4864700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01a0090 con 0x7f67f01056c0 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67edd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67f01056c0 0x7f67f0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67edd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67f01056c0 0x7f67f0198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:47274/0 (socket says 172.21.15.155:47274) 2024-03-23T13:12:43.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67edd9b700 1 -- 172.21.15.155:0/4091855691 learned_addr learned my addr 172.21.15.155:0/4091855691 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67ee59c700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f0102d90 0x7f67f0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67eed9d700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67f0107ff0 0x7f67f019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67edd9b700 1 -- 172.21.15.155:0/4091855691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67f0107ff0 msgr2=0x7f67f019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67edd9b700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67f0107ff0 0x7f67f019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67edd9b700 1 -- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f0102d90 msgr2=0x7f67f0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67edd9b700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f0102d90 0x7f67f0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67edd9b700 1 -- 172.21.15.155:0/4091855691 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e00095f0 con 0x7f67f01056c0 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67ee59c700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f0102d90 0x7f67f0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67eed9d700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67f0107ff0 0x7f67f019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67edd9b700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67f01056c0 0x7f67f0198770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f67e400d980 tx=0x7f67e400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67df7fe700 1 -- 172.21.15.155:0/4091855691 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4009b40 con 0x7f67f01056c0 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67f4864700 1 -- 172.21.15.155:0/4091855691 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f01a0370 con 0x7f67f01056c0 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67df7fe700 1 -- 172.21.15.155:0/4091855691 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e400b6e0 con 0x7f67f01056c0 2024-03-23T13:12:43.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67df7fe700 1 -- 172.21.15.155:0/4091855691 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e400f620 con 0x7f67f01056c0 2024-03-23T13:12:43.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.777+0000 7f67f4864700 1 -- 172.21.15.155:0/4091855691 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f01a08c0 con 0x7f67f01056c0 2024-03-23T13:12:43.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.778+0000 7f67f4864700 1 -- 172.21.15.155:0/4091855691 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f00fe2e0 con 0x7f67f01056c0 2024-03-23T13:12:43.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.780+0000 7f67df7fe700 1 -- 172.21.15.155:0/4091855691 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 15) ==== 136699+0+0 (secure 0 0 0) 0x7f67e4010460 con 0x7f67f01056c0 2024-03-23T13:12:43.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.780+0000 7f67df7fe700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f67d80a20c0 0x7f67d80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:12:43.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.780+0000 7f67df7fe700 1 -- 172.21.15.155:0/4091855691 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f67e40cc2e0 con 0x7f67f01056c0 2024-03-23T13:12:43.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.780+0000 7f67ee59c700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f67d80a20c0 0x7f67d80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:12:43.776 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.781+0000 7f67ee59c700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f67d80a20c0 0x7f67d80a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67e00095c0 tx=0x7f67e0009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:12:43.778 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.784+0000 7f67df7fe700 1 -- 172.21.15.155:0/4091855691 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f67e4096b00 con 0x7f67f01056c0 2024-03-23T13:12:43.980 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:43 smithi155 ceph-mon[36157]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:43.980 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:43 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi130 2024-03-23T13:12:43.980 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:43 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi130 to 17080M 2024-03-23T13:12:43.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:12:43.986+0000 7f67f4864700 1 -- 172.21.15.155:0/4091855691 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f67f002db60 con 0x7f67d80a20c0 2024-03-23T13:12:44.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:43 smithi007 ceph-mon[32184]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:44.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:43 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi130 2024-03-23T13:12:44.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:43 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi130 to 17080M 2024-03-23T13:12:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:43 smithi130 ceph-mon[36225]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T13:12:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:43 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi130 2024-03-23T13:12:44.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:43 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi130 to 17080M 2024-03-23T13:12:44.846 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:44 smithi155 ceph-mon[36157]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T13:12:44.846 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:44 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:12:44.846 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:44 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:12:44.846 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:44 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:44 smithi007 ceph-mon[32184]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T13:12:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:44 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:12:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:44 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:12:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:44 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:45.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:44 smithi130 ceph-mon[36225]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T13:12:45.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:44 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:12:45.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:44 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:12:45.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:44 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:46.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:45 smithi007 ceph-mon[32184]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:46.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:45 smithi007 ceph-mon[32184]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:12:46.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:45 smithi130 ceph-mon[36225]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:46.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:45 smithi130 ceph-mon[36225]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:12:46.282 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:45 smithi155 ceph-mon[36157]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:46.282 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:45 smithi155 ceph-mon[36157]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:12:47.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:46 smithi155 ceph-mon[36157]: from='client.? 172.21.15.155:0/1077533398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2013fd43-79f2-420b-8518-046fce065cd7"}]: dispatch 2024-03-23T13:12:47.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:46 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2013fd43-79f2-420b-8518-046fce065cd7"}]: dispatch 2024-03-23T13:12:47.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:46 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2013fd43-79f2-420b-8518-046fce065cd7"}]': finished 2024-03-23T13:12:47.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:46 smithi155 ceph-mon[36157]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T13:12:47.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:12:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:46 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/1077533398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2013fd43-79f2-420b-8518-046fce065cd7"}]: dispatch 2024-03-23T13:12:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:46 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2013fd43-79f2-420b-8518-046fce065cd7"}]: dispatch 2024-03-23T13:12:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:46 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2013fd43-79f2-420b-8518-046fce065cd7"}]': finished 2024-03-23T13:12:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:46 smithi007 ceph-mon[32184]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T13:12:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:12:47.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:46 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/1077533398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2013fd43-79f2-420b-8518-046fce065cd7"}]: dispatch 2024-03-23T13:12:47.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:46 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2013fd43-79f2-420b-8518-046fce065cd7"}]: dispatch 2024-03-23T13:12:47.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:46 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2013fd43-79f2-420b-8518-046fce065cd7"}]': finished 2024-03-23T13:12:47.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:46 smithi130 ceph-mon[36225]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T13:12:47.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:12:48.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:47 smithi007 ceph-mon[32184]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:48.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:47 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/622643473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:12:48.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:47 smithi130 ceph-mon[36225]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:48.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:47 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/622643473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:12:48.282 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:47 smithi155 ceph-mon[36157]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:48.282 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:47 smithi155 ceph-mon[36157]: from='client.? 172.21.15.155:0/622643473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:12:50.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:49 smithi007 ceph-mon[32184]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:50.187 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:49 smithi155 ceph-mon[36157]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:50.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:49 smithi130 ceph-mon[36225]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:52.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:51 smithi155 ceph-mon[36157]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:52.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:51 smithi007 ceph-mon[32184]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:52.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:51 smithi130 ceph-mon[36225]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:54.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:53 smithi007 ceph-mon[32184]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:54.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T13:12:54.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:54.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:53 smithi130 ceph-mon[36225]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:54.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T13:12:54.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:54.202 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:53 smithi155 ceph-mon[36157]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:54.202 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T13:12:54.203 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:12:55.065 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:54 smithi155 ceph-mon[36157]: Deploying daemon osd.2 on smithi155 2024-03-23T13:12:55.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:54 smithi007 ceph-mon[32184]: Deploying daemon osd.2 on smithi155 2024-03-23T13:12:55.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:54 smithi130 ceph-mon[36225]: Deploying daemon osd.2 on smithi155 2024-03-23T13:12:56.021 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:55 smithi155 ceph-mon[36157]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:56.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:55 smithi007 ceph-mon[32184]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:56.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:55 smithi130 ceph-mon[36225]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:57 smithi007 ceph-mon[32184]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:58.179 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:57 smithi155 ceph-mon[36157]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:12:58.188 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:57 smithi130 ceph-mon[36225]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:00.032 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:59 smithi155 ceph-mon[36157]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:00.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:00.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:00.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:12:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:59 smithi007 ceph-mon[32184]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:12:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:00.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:59 smithi130 ceph-mon[36225]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:00.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:00.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:00.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:12:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:02.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:01 smithi007 ceph-mon[32184]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:02.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:01 smithi130 ceph-mon[36225]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:02.205 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:01 smithi155 ceph-mon[36157]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:02.246 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.252+0000 7f67df7fe700 1 -- 172.21.15.155:0/4091855691 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f67f002db60 con 0x7f67d80a20c0 2024-03-23T13:13:02.246 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 2 on host 'smithi155' 2024-03-23T13:13:02.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.254+0000 7f67f4864700 1 -- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f67d80a20c0 msgr2=0x7f67d80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:02.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.254+0000 7f67f4864700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f67d80a20c0 0x7f67d80a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67e00095c0 tx=0x7f67e0009510 comp rx=0 tx=0).stop 2024-03-23T13:13:02.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.254+0000 7f67f4864700 1 -- 172.21.15.155:0/4091855691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67f01056c0 msgr2=0x7f67f0198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:02.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.254+0000 7f67f4864700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67f01056c0 0x7f67f0198770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f67e400d980 tx=0x7f67e400dd40 comp rx=0 tx=0).stop 2024-03-23T13:13:02.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.254+0000 7f67f4864700 1 -- 172.21.15.155:0/4091855691 shutdown_connections 2024-03-23T13:13:02.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.254+0000 7f67f4864700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f67d80a20c0 0x7f67d80a4580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:02.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.254+0000 7f67f4864700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67f0107ff0 0x7f67f019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:02.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.254+0000 7f67f4864700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f0102d90 0x7f67f0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:02.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.254+0000 7f67f4864700 1 --2- 172.21.15.155:0/4091855691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67f01056c0 0x7f67f0198770 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:02.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.254+0000 7f67f4864700 1 -- 172.21.15.155:0/4091855691 >> 172.21.15.155:0/4091855691 conn(0x7f67f00fc760 msgr2=0x7f67f01036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:13:02.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.255+0000 7f67f4864700 1 -- 172.21.15.155:0/4091855691 shutdown_connections 2024-03-23T13:13:02.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:02.255+0000 7f67f4864700 1 -- 172.21.15.155:0/4091855691 wait complete. 2024-03-23T13:13:02.802 DEBUG:teuthology.orchestra.run.smithi155:osd.2> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.2.service 2024-03-23T13:13:02.804 INFO:tasks.cephadm:Deploying osd.3 on smithi007 with /dev/vg_nvme/lv_3... 2024-03-23T13:13:02.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T13:13:03.032 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:02 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2[41320]: 2024-03-23T13:13:02.877+0000 7f8d36b397c0 -1 osd.2 0 log_to_monitors true 2024-03-23T13:13:03.059 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:13:03.468 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:03 smithi007 ceph-mon[32184]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:03.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:03 smithi007 ceph-mon[32184]: from='osd.2 [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T13:13:03.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:03 smithi007 ceph-mon[32184]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T13:13:03.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:03 smithi155 ceph-mon[36157]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:03.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:03 smithi155 ceph-mon[36157]: from='osd.2 [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T13:13:03.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:03 smithi155 ceph-mon[36157]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T13:13:03.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:03 smithi130 ceph-mon[36225]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:03.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:03 smithi130 ceph-mon[36225]: from='osd.2 [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T13:13:03.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:03 smithi130 ceph-mon[36225]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T13:13:03.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:03.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:04.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:04 smithi130 ceph-mon[36225]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T13:13:04.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:04 smithi130 ceph-mon[36225]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T13:13:04.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:04 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:04.964 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:04 smithi130 ceph-mon[36225]: from='osd.2 [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:13:04.965 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:04 smithi130 ceph-mon[36225]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:13:05.030 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:04 smithi155 ceph-mon[36157]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T13:13:05.030 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:04 smithi155 ceph-mon[36157]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T13:13:05.030 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:04 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:05.030 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:04 smithi155 ceph-mon[36157]: from='osd.2 [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:13:05.030 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:04 smithi155 ceph-mon[36157]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:13:05.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:04 smithi007 ceph-mon[32184]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T13:13:05.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:04 smithi007 ceph-mon[32184]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T13:13:05.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:04 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:05.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:04 smithi007 ceph-mon[32184]: from='osd.2 [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:13:05.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:04 smithi007 ceph-mon[32184]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:13:05.162 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T13:13:05.162 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T13:13:05.162 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-23T13:13:05.162 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-23T13:13:05.162 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00869295 s, 1.2 GB/s 2024-03-23T13:13:05.162 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-23T13:13:05.282 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:05 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2[41320]: 2024-03-23T13:13:05.277+0000 7f8d2c19e700 -1 osd.2 0 waiting for initial osdmap 2024-03-23T13:13:05.282 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:05 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2[41320]: 2024-03-23T13:13:05.283+0000 7f8d23f89700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:13:05.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_3 2024-03-23T13:13:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:13:06.172 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:05 smithi155 ceph-mon[36157]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:06.172 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:05 smithi155 ceph-mon[36157]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:13:06.172 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:05 smithi155 ceph-mon[36157]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T13:13:06.172 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:05 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:06.172 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:05 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:06.172 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:05 smithi155 ceph-mon[36157]: from='osd.2 ' entity='osd.2' 2024-03-23T13:13:06.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:05 smithi130 ceph-mon[36225]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:06.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:05 smithi130 ceph-mon[36225]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:13:06.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:05 smithi130 ceph-mon[36225]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T13:13:06.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:05 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:06.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:05 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:06.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:05 smithi130 ceph-mon[36225]: from='osd.2 ' entity='osd.2' 2024-03-23T13:13:06.253 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:05 smithi007 ceph-mon[32184]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:06.253 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:05 smithi007 ceph-mon[32184]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:13:06.253 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:05 smithi007 ceph-mon[32184]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T13:13:06.253 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:06.253 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:06.254 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:05 smithi007 ceph-mon[32184]: from='osd.2 ' entity='osd.2' 2024-03-23T13:13:07.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:13:07.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:13:07.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: osd.2 [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] boot 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:07.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:07.191 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:07.191 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: osd.2 [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] boot 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:07.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:07.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:07.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:07.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:07.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: osd.2 [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] boot 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:07.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:07.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:07.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.810+0000 7fa1cb55b700 1 -- 172.21.15.7:0/1271048306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1c410a150 msgr2=0x7fa1c410c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.810+0000 7fa1cb55b700 1 --2- 172.21.15.7:0/1271048306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1c410a150 0x7fa1c410c5c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400b0a0 tx=0x7fa1b400b3b0 comp rx=0 tx=0).stop 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.810+0000 7fa1cb55b700 1 -- 172.21.15.7:0/1271048306 shutdown_connections 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.810+0000 7fa1cb55b700 1 --2- 172.21.15.7:0/1271048306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1c410a150 0x7fa1c410c5c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.810+0000 7fa1cb55b700 1 --2- 172.21.15.7:0/1271048306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c40761b0 0x7fa1c4074610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.810+0000 7fa1cb55b700 1 --2- 172.21.15.7:0/1271048306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1c4074be0 0x7fa1c4075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.810+0000 7fa1cb55b700 1 -- 172.21.15.7:0/1271048306 >> 172.21.15.7:0/1271048306 conn(0x7fa1c40fd810 msgr2=0x7fa1c40ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.810+0000 7fa1cb55b700 1 -- 172.21.15.7:0/1271048306 shutdown_connections 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.810+0000 7fa1cb55b700 1 -- 172.21.15.7:0/1271048306 wait complete. 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1cb55b700 1 Processor -- start 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1cb55b700 1 -- start start 2024-03-23T13:13:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1cb55b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c4074be0 0x7fa1c419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1cb55b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1c40761b0 0x7fa1c419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1cb55b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1c410a150 0x7fa1c41a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1cb55b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c41a2080 con 0x7fa1c410a150 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1cb55b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c41a21c0 con 0x7fa1c40761b0 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1cb55b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c41a2350 con 0x7fa1c4074be0 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1c9af8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1c410a150 0x7fa1c41a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1c9af8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1c410a150 0x7fa1c41a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58132/0 (socket says 172.21.15.7:58132) 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1c9af8700 1 -- 172.21.15.7:0/4204154506 learned_addr learned my addr 172.21.15.7:0/4204154506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1c8af6700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1c40761b0 0x7fa1c419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1c92f7700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c4074be0 0x7fa1c419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1c9af8700 1 -- 172.21.15.7:0/4204154506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c4074be0 msgr2=0x7fa1c419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1c9af8700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c4074be0 0x7fa1c419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1c9af8700 1 -- 172.21.15.7:0/4204154506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1c40761b0 msgr2=0x7fa1c419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1c9af8700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1c40761b0 0x7fa1c419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.811+0000 7fa1c9af8700 1 -- 172.21.15.7:0/4204154506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b4009d00 con 0x7fa1c410a150 2024-03-23T13:13:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.812+0000 7fa1c9af8700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1c410a150 0x7fa1c41a1ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40093f0 tx=0x7fa1b4009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:13:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.812+0000 7fa1c27fc700 1 -- 172.21.15.7:0/4204154506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b400e040 con 0x7fa1c410a150 2024-03-23T13:13:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.812+0000 7fa1cb55b700 1 -- 172.21.15.7:0/4204154506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c41a25d0 con 0x7fa1c410a150 2024-03-23T13:13:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.812+0000 7fa1c27fc700 1 -- 172.21.15.7:0/4204154506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b401d960 con 0x7fa1c410a150 2024-03-23T13:13:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.812+0000 7fa1c27fc700 1 -- 172.21.15.7:0/4204154506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b401bac0 con 0x7fa1c410a150 2024-03-23T13:13:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.812+0000 7fa1cb55b700 1 -- 172.21.15.7:0/4204154506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c41a2b10 con 0x7fa1c410a150 2024-03-23T13:13:07.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.813+0000 7fa1cb55b700 1 -- 172.21.15.7:0/4204154506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c4108f60 con 0x7fa1c410a150 2024-03-23T13:13:07.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.814+0000 7fa1c27fc700 1 -- 172.21.15.7:0/4204154506 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 136699+0+0 (secure 0 0 0) 0x7fa1b401dad0 con 0x7fa1c410a150 2024-03-23T13:13:07.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.814+0000 7fa1c27fc700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa1ac0a20c0 0x7fa1ac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:07.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.814+0000 7fa1c92f7700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa1ac0a20c0 0x7fa1ac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:07.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.814+0000 7fa1c27fc700 1 -- 172.21.15.7:0/4204154506 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7fa1b40ce350 con 0x7fa1c410a150 2024-03-23T13:13:07.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.815+0000 7fa1c92f7700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa1ac0a20c0 0x7fa1ac0a4580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc006fd0 tx=0x7fa1bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:13:07.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:07.819+0000 7fa1c27fc700 1 -- 172.21.15.7:0/4204154506 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa1b4098940 con 0x7fa1c410a150 2024-03-23T13:13:08.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:08.035+0000 7fa1cb55b700 1 -- 172.21.15.7:0/4204154506 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa1c402db60 con 0x7fa1ac0a20c0 2024-03-23T13:13:08.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:07 smithi130 ceph-mon[36225]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:08.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:07 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi155 2024-03-23T13:13:08.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:07 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi155 to 17080M 2024-03-23T13:13:08.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:07 smithi155 ceph-mon[36157]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:08.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:07 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi155 2024-03-23T13:13:08.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:07 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi155 to 17080M 2024-03-23T13:13:08.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:07 smithi007 ceph-mon[32184]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T13:13:08.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:07 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi155 2024-03-23T13:13:08.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:07 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi155 to 17080M 2024-03-23T13:13:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:08 smithi007 ceph-mon[32184]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T13:13:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:08 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T13:13:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:08 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:13:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:08 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:13:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:08 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:09.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:08 smithi130 ceph-mon[36225]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T13:13:09.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:08 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T13:13:09.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:08 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:13:09.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:08 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:13:09.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:08 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:09.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:08 smithi155 ceph-mon[36157]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T13:13:09.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:08 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T13:13:09.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:08 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:13:09.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:08 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:13:09.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:08 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:09.890 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:09 smithi007 ceph-mon[32184]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:09.891 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:09 smithi007 ceph-mon[32184]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:13:09.891 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:09 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T13:13:09.891 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:09 smithi007 ceph-mon[32184]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T13:13:09.891 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:09 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T13:13:10.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:09 smithi130 ceph-mon[36225]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:10.200 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:09 smithi130 ceph-mon[36225]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:13:10.201 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:09 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T13:13:10.201 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:09 smithi130 ceph-mon[36225]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T13:13:10.201 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:09 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T13:13:10.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:09 smithi155 ceph-mon[36157]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:10.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:09 smithi155 ceph-mon[36157]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:13:10.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:09 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T13:13:10.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:09 smithi155 ceph-mon[36157]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T13:13:10.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:09 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T13:13:10.477 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:13:10 smithi007 sudo[51678]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T13:13:10.477 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:13:10 smithi007 sudo[51678]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:10.477 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:13:10 smithi007 sudo[51678]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:10.477 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:13:10 smithi007 sudo[51678]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:10.821 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:13:10 smithi007 sudo[51701]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T13:13:10.822 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:13:10 smithi007 sudo[51701]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:10.822 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:13:10 smithi007 sudo[51701]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:10.822 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:13:10 smithi007 sudo[51701]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:11.002 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:13:10 smithi130 sudo[44989]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T13:13:11.002 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:13:10 smithi130 sudo[44989]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:11.002 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:13:10 smithi130 sudo[44989]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:11.003 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:13:10 smithi130 sudo[44989]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:11.003 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T13:13:11.003 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:10 smithi130 ceph-mon[36225]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T13:13:11.003 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:10 smithi130 ceph-mon[36225]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:11.003 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:11.003 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:10 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1598366276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a418507-c9a0-4945-aca6-43c156f4f96b"}]: dispatch 2024-03-23T13:13:11.003 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:10 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1598366276' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a418507-c9a0-4945-aca6-43c156f4f96b"}]': finished 2024-03-23T13:13:11.003 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:10 smithi130 ceph-mon[36225]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T13:13:11.003 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:11.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T13:13:11.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:10 smithi007 ceph-mon[32184]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T13:13:11.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:10 smithi007 ceph-mon[32184]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:11.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:11.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:10 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1598366276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a418507-c9a0-4945-aca6-43c156f4f96b"}]: dispatch 2024-03-23T13:13:11.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:10 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1598366276' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a418507-c9a0-4945-aca6-43c156f4f96b"}]': finished 2024-03-23T13:13:11.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:10 smithi007 ceph-mon[32184]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T13:13:11.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:11.267 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T13:13:11.267 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:10 smithi155 ceph-mon[36157]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T13:13:11.267 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:10 smithi155 ceph-mon[36157]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:11.267 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:11.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:10 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1598366276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a418507-c9a0-4945-aca6-43c156f4f96b"}]: dispatch 2024-03-23T13:13:11.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:10 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1598366276' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a418507-c9a0-4945-aca6-43c156f4f96b"}]': finished 2024-03-23T13:13:11.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:10 smithi155 ceph-mon[36157]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T13:13:11.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:11.439 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:13:11 smithi130 sudo[44992]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T13:13:11.439 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:13:11 smithi130 sudo[44992]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:11.439 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:13:11 smithi130 sudo[44992]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:11.439 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:13:11 smithi130 sudo[44992]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:11.515+0000 7fa1c27fc700 1 -- 172.21.15.7:0/4204154506 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fa1b4098d10 con 0x7fa1c410a150 2024-03-23T13:13:11.521 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:11 smithi155 sudo[45238]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T13:13:11.521 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:11 smithi155 sudo[45238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:11.521 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:11 smithi155 sudo[45238]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:11.521 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:11 smithi155 sudo[45238]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:11.783 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:11 smithi155 sudo[45241]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T13:13:11.783 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:11 smithi155 sudo[45241]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:11.783 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:11 smithi155 sudo[45241]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:11.783 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:13:11 smithi155 sudo[45241]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:12.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:11 smithi007 sudo[51808]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T13:13:12.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:11 smithi007 sudo[51808]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:12.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:11 smithi007 sudo[51808]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:12.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:11 smithi007 sudo[51808]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:12.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:12 smithi007 sudo[51994]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T13:13:12.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:12 smithi007 sudo[51994]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:12.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:12 smithi007 sudo[51994]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:12.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:12 smithi007 sudo[51994]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:12.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:12.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:12 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2361538941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:13:12.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:12 smithi007 ceph-mon[32184]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T13:13:12.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:12.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:12 smithi007 ceph-mon[32184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T13:13:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2361538941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:13:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 ceph-mon[36157]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T13:13:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 ceph-mon[36157]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T13:13:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 sudo[45244]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T13:13:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 sudo[45244]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 sudo[45244]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:12.323 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 sudo[45244]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:12.439 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:12.439 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:12 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2361538941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:13:12.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:12 smithi130 ceph-mon[36225]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T13:13:12.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:12.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:12 smithi130 ceph-mon[36225]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T13:13:12.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 sudo[45247]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T13:13:12.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 sudo[45247]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:12.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 sudo[45247]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:12.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:12 smithi155 sudo[45247]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:13.048 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:12 smithi130 sudo[44995]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T13:13:13.048 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:12 smithi130 sudo[44995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:13.049 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:12 smithi130 sudo[44995]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:13.049 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:12 smithi130 sudo[44995]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:13.049 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 sudo[44998]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 sudo[44998]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 sudo[44998]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 sudo[44998]: pam_unix(sudo:session): session closed for user root 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:13:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:13:13.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T13:13:13.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T13:13:13.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:13:13.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:13:13.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:13:13.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:13 smithi130 ceph-mon[36225]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T13:13:13.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:13.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T13:13:13.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:13:13.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:13:13.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:13:13.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T13:13:13.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:13:13.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:13:13.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:13:13.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T13:13:13.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T13:13:13.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:13:13.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:13:13.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:13:13.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:13 smithi155 ceph-mon[36157]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T13:13:13.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:13.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T13:13:13.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:13:13.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:13:13.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:13:13.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T13:13:13.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:13:13.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:13:13.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:13:13.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T13:13:13.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T13:13:13.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T13:13:13.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T13:13:13.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T13:13:13.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:13 smithi007 ceph-mon[32184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T13:13:15.336 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:15 smithi155 ceph-mon[36157]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:15.386 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:15 smithi130 ceph-mon[36225]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:15.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:15 smithi007 ceph-mon[32184]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:17.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:17 smithi007 ceph-mon[32184]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:17.439 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:17 smithi130 ceph-mon[36225]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:17.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:17 smithi155 ceph-mon[36157]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:19.439 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:19 smithi130 ceph-mon[36225]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-03-23T13:13:19.439 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T13:13:19.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:19.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:19 smithi130 ceph-mon[36225]: Deploying daemon osd.3 on smithi007 2024-03-23T13:13:19.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:19 smithi155 ceph-mon[36157]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-03-23T13:13:19.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T13:13:19.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:19.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:19 smithi155 ceph-mon[36157]: Deploying daemon osd.3 on smithi007 2024-03-23T13:13:19.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:19 smithi007 ceph-mon[32184]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-03-23T13:13:19.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T13:13:19.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:19.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:19 smithi007 ceph-mon[32184]: Deploying daemon osd.3 on smithi007 2024-03-23T13:13:21.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:21 smithi007 ceph-mon[32184]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-23T13:13:21.439 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:21 smithi130 ceph-mon[36225]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-23T13:13:21.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:21 smithi155 ceph-mon[36157]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-23T13:13:23.176 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:23 smithi007 ceph-mon[32184]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-23T13:13:23.439 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:23 smithi130 ceph-mon[36225]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-23T13:13:23.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:23 smithi155 ceph-mon[36157]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-23T13:13:25.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:25.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:25.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:25.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:25.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:25.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:25.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:25.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:25.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:26.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:25 smithi155 ceph-mon[36157]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-23T13:13:26.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:25 smithi007 ceph-mon[32184]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-23T13:13:26.189 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:25 smithi130 ceph-mon[36225]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-23T13:13:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.206+0000 7fa1c27fc700 1 -- 172.21.15.7:0/4204154506 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa1c402db60 con 0x7fa1ac0a20c0 2024-03-23T13:13:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 3 on host 'smithi007' 2024-03-23T13:13:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.208+0000 7fa1cb55b700 1 -- 172.21.15.7:0/4204154506 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa1ac0a20c0 msgr2=0x7fa1ac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.208+0000 7fa1cb55b700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa1ac0a20c0 0x7fa1ac0a4580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc006fd0 tx=0x7fa1bc008040 comp rx=0 tx=0).stop 2024-03-23T13:13:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.208+0000 7fa1cb55b700 1 -- 172.21.15.7:0/4204154506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1c410a150 msgr2=0x7fa1c41a1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.208+0000 7fa1cb55b700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1c410a150 0x7fa1c41a1ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40093f0 tx=0x7fa1b4009420 comp rx=0 tx=0).stop 2024-03-23T13:13:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.208+0000 7fa1cb55b700 1 -- 172.21.15.7:0/4204154506 shutdown_connections 2024-03-23T13:13:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.208+0000 7fa1cb55b700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa1ac0a20c0 0x7fa1ac0a4580 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.208+0000 7fa1cb55b700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1c410a150 0x7fa1c41a1ab0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.208+0000 7fa1cb55b700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c4074be0 0x7fa1c419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.208+0000 7fa1cb55b700 1 --2- 172.21.15.7:0/4204154506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1c40761b0 0x7fa1c419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.208+0000 7fa1cb55b700 1 -- 172.21.15.7:0/4204154506 >> 172.21.15.7:0/4204154506 conn(0x7fa1c40fd810 msgr2=0x7fa1c41060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:13:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.209+0000 7fa1cb55b700 1 -- 172.21.15.7:0/4204154506 shutdown_connections 2024-03-23T13:13:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:13:27.209+0000 7fa1cb55b700 1 -- 172.21.15.7:0/4204154506 wait complete. 2024-03-23T13:13:27.820 DEBUG:teuthology.orchestra.run.smithi007:osd.3> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.3.service 2024-03-23T13:13:27.823 INFO:tasks.cephadm:Deploying osd.4 on smithi130 with /dev/vg_nvme/lv_3... 2024-03-23T13:13:27.823 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T13:13:28.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:27 smithi155 ceph-mon[36157]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T13:13:28.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:27 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:28.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:27 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:28.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:27 smithi155 ceph-mon[36157]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T13:13:28.068 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:13:28.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:27 smithi007 ceph-mon[32184]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T13:13:28.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:27 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:28.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:27 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:28.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:27 smithi007 ceph-mon[32184]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T13:13:28.173 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:27 smithi130 ceph-mon[36225]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T13:13:28.173 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:27 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:28.173 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:27 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:28.173 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:27 smithi130 ceph-mon[36225]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T13:13:29.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:28 smithi007 ceph-mon[32184]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T13:13:29.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:29.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:29.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:28 smithi007 ceph-mon[32184]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T13:13:29.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:28 smithi007 ceph-mon[32184]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T13:13:29.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:28 smithi007 ceph-mon[32184]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:13:29.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:29.161 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:28 smithi130 ceph-mon[36225]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T13:13:29.161 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:29.161 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:29.162 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:28 smithi130 ceph-mon[36225]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T13:13:29.162 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:28 smithi130 ceph-mon[36225]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T13:13:29.162 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:28 smithi130 ceph-mon[36225]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:13:29.162 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:29.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:28 smithi155 ceph-mon[36157]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T13:13:29.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:29.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:29.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:28 smithi155 ceph-mon[36157]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T13:13:29.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:28 smithi155 ceph-mon[36157]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T13:13:29.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:28 smithi155 ceph-mon[36157]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:13:29.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:29.976 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:13:29 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3[53764]: 2024-03-23T13:13:29.687+0000 7f2be4bc6700 -1 osd.3 0 waiting for initial osdmap 2024-03-23T13:13:29.977 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:13:29 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3[53764]: 2024-03-23T13:13:29.692+0000 7f2bdc9b1700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:13:30.072 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T13:13:30.072 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T13:13:30.072 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-03-23T13:13:30.072 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-03-23T13:13:30.073 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125166 s, 838 MB/s 2024-03-23T13:13:30.073 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-03-23T13:13:30.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:30 smithi007 ceph-mon[32184]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:13:30.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:30 smithi007 ceph-mon[32184]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T13:13:30.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:30 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:30.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:30 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:30.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:30 smithi007 ceph-mon[32184]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' 2024-03-23T13:13:30.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:30 smithi155 ceph-mon[36157]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:13:30.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:30 smithi155 ceph-mon[36157]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T13:13:30.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:30 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:30.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:30 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:30.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:30 smithi155 ceph-mon[36157]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' 2024-03-23T13:13:30.535 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:30 smithi130 ceph-mon[36225]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:13:30.535 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:30 smithi130 ceph-mon[36225]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T13:13:30.535 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:30 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:30.535 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:30 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:30.535 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:30 smithi130 ceph-mon[36225]: from='osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179]' entity='osd.3' 2024-03-23T13:13:30.670 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi130:vg_nvme/lv_3 2024-03-23T13:13:30.935 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:13:31.308 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:31 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:13:31.308 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:31 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:13:31.308 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:31 smithi130 ceph-mon[36225]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:31.308 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:31 smithi130 ceph-mon[36225]: osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] boot 2024-03-23T13:13:31.308 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:31 smithi130 ceph-mon[36225]: osdmap e25: 4 total, 4 up, 4 in 2024-03-23T13:13:31.308 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:31 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:31.308 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:31 smithi130 ceph-mon[36225]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T13:13:31.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:31 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:13:31.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:31 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:13:31.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:31 smithi155 ceph-mon[36157]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:31.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:31 smithi155 ceph-mon[36157]: osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] boot 2024-03-23T13:13:31.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:31 smithi155 ceph-mon[36157]: osdmap e25: 4 total, 4 up, 4 in 2024-03-23T13:13:31.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:31 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:31.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:31 smithi155 ceph-mon[36157]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T13:13:31.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:31 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:13:31.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:31 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:13:31.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:31 smithi007 ceph-mon[32184]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-23T13:13:31.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:31 smithi007 ceph-mon[32184]: osd.3 [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] boot 2024-03-23T13:13:31.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:31 smithi007 ceph-mon[32184]: osdmap e25: 4 total, 4 up, 4 in 2024-03-23T13:13:31.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:31 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T13:13:31.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:31 smithi007 ceph-mon[32184]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T13:13:32.338 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi007 2024-03-23T13:13:32.338 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.338 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.338 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:32.338 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:32.338 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi007 to 8546M 2024-03-23T13:13:32.339 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.339 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:32.339 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:32.339 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.339 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:32.339 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:32.339 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:32.339 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi007 2024-03-23T13:13:32.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi007 to 8546M 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:32.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.571+0000 7fe842f9e700 1 -- 172.21.15.130:0/2110187948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c103dc0 msgr2=0x7fe83c10a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.571+0000 7fe842f9e700 1 --2- 172.21.15.130:0/2110187948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c103dc0 0x7fe83c10a670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe83400b0d0 tx=0x7fe83400b3e0 comp rx=0 tx=0).stop 2024-03-23T13:13:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.571+0000 7fe842f9e700 1 -- 172.21.15.130:0/2110187948 shutdown_connections 2024-03-23T13:13:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.571+0000 7fe842f9e700 1 --2- 172.21.15.130:0/2110187948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c103dc0 0x7fe83c10a670 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.571+0000 7fe842f9e700 1 --2- 172.21.15.130:0/2110187948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c102220 0x7fe83c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.571+0000 7fe842f9e700 1 --2- 172.21.15.130:0/2110187948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe83c103420 0x7fe83c103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 -- 172.21.15.130:0/2110187948 >> 172.21.15.130:0/2110187948 conn(0x7fe83c0fd7e0 msgr2=0x7fe83c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:13:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 -- 172.21.15.130:0/2110187948 shutdown_connections 2024-03-23T13:13:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 -- 172.21.15.130:0/2110187948 wait complete. 2024-03-23T13:13:32.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 Processor -- start 2024-03-23T13:13:32.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 -- start start 2024-03-23T13:13:32.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c102220 0x7fe83c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:32.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c103420 0x7fe83c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:32.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe83c103dc0 0x7fe83c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:32.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c19fe70 con 0x7fe83c103420 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c19ffb0 con 0x7fe83c102220 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe842f9e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1a0140 con 0x7fe83c103dc0 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe840d3a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c102220 0x7fe83c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe840d3a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c102220 0x7fe83c1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:52196/0 (socket says 172.21.15.130:52196) 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe840d3a700 1 -- 172.21.15.130:0/158484421 learned_addr learned my addr 172.21.15.130:0/158484421 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.572+0000 7fe83bfff700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c103420 0x7fe83c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe840d3a700 1 -- 172.21.15.130:0/158484421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe83c103dc0 msgr2=0x7fe83c19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe840d3a700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe83c103dc0 0x7fe83c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe840d3a700 1 -- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c103420 msgr2=0x7fe83c198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe840d3a700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c103420 0x7fe83c198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe840d3a700 1 -- 172.21.15.130:0/158484421 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe834009d00 con 0x7fe83c102220 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe83bfff700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c103420 0x7fe83c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe840d3a700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c102220 0x7fe83c1982e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe83000d980 tx=0x7fe83000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:13:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe839ffb700 1 -- 172.21.15.130:0/158484421 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83000bb60 con 0x7fe83c102220 2024-03-23T13:13:32.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe842f9e700 1 -- 172.21.15.130:0/158484421 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c1a0420 con 0x7fe83c102220 2024-03-23T13:13:32.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe839ffb700 1 -- 172.21.15.130:0/158484421 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe830007620 con 0x7fe83c102220 2024-03-23T13:13:32.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe839ffb700 1 -- 172.21.15.130:0/158484421 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83000f620 con 0x7fe83c102220 2024-03-23T13:13:32.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.573+0000 7fe842f9e700 1 -- 172.21.15.130:0/158484421 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c1a0970 con 0x7fe83c102220 2024-03-23T13:13:32.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.574+0000 7fe842f9e700 1 -- 172.21.15.130:0/158484421 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83c109580 con 0x7fe83c102220 2024-03-23T13:13:32.572 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.575+0000 7fe839ffb700 1 -- 172.21.15.130:0/158484421 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fe830010460 con 0x7fe83c102220 2024-03-23T13:13:32.572 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.576+0000 7fe839ffb700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe82c0a2150 0x7fe82c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:32.572 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.576+0000 7fe839ffb700 1 -- 172.21.15.130:0/158484421 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fe8300cc9c0 con 0x7fe83c102220 2024-03-23T13:13:32.572 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.576+0000 7fe83bfff700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe82c0a2150 0x7fe82c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:32.572 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.576+0000 7fe83bfff700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe82c0a2150 0x7fe82c0a4610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe8280099d0 tx=0x7fe828008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:13:32.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi007 2024-03-23T13:13:32.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:32.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi007 to 8546M 2024-03-23T13:13:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:32.575 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.580+0000 7fe839ffb700 1 -- 172.21.15.130:0/158484421 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fe830096b90 con 0x7fe83c102220 2024-03-23T13:13:32.780 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:32.785+0000 7fe842f9e700 1 -- 172.21.15.130:0/158484421 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe83c02dc00 con 0x7fe82c0a2150 2024-03-23T13:13:33.502 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:33 smithi130 ceph-mon[36225]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:33.502 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:33 smithi130 ceph-mon[36225]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T13:13:33.502 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:33 smithi130 ceph-mon[36225]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:13:33.502 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:33 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:13:33.502 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:33 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:13:33.502 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:33 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:33.502 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:33 smithi130 ceph-mon[36225]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T13:13:33.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:33 smithi155 ceph-mon[36157]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:33.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:33 smithi155 ceph-mon[36157]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T13:13:33.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:33 smithi155 ceph-mon[36157]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:13:33.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:33 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:13:33.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:33 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:13:33.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:33 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:33.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:33 smithi155 ceph-mon[36157]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T13:13:33.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:33 smithi007 ceph-mon[32184]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:33.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:33 smithi007 ceph-mon[32184]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T13:13:33.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:33 smithi007 ceph-mon[32184]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:13:33.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:33 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:13:33.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:33 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:13:33.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:33 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:33.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:33 smithi007 ceph-mon[32184]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T13:13:35.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:35 smithi155 ceph-mon[36157]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:13:35.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:35 smithi155 ceph-mon[36157]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T13:13:35.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:35 smithi007 ceph-mon[32184]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:13:35.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:35 smithi007 ceph-mon[32184]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T13:13:35.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:35 smithi130 ceph-mon[36225]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:13:35.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:35 smithi130 ceph-mon[36225]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T13:13:36.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:36 smithi155 ceph-mon[36157]: from='client.? 172.21.15.130:0/3230312749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "258432a8-3995-4878-928f-1bc7cc486699"}]: dispatch 2024-03-23T13:13:36.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:36 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "258432a8-3995-4878-928f-1bc7cc486699"}]: dispatch 2024-03-23T13:13:36.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:36 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "258432a8-3995-4878-928f-1bc7cc486699"}]': finished 2024-03-23T13:13:36.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:36 smithi155 ceph-mon[36157]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T13:13:36.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:36 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:36.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:36 smithi155 ceph-mon[36157]: from='client.? 172.21.15.130:0/622898459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:13:36.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:36 smithi007 ceph-mon[32184]: from='client.? 172.21.15.130:0/3230312749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "258432a8-3995-4878-928f-1bc7cc486699"}]: dispatch 2024-03-23T13:13:36.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:36 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "258432a8-3995-4878-928f-1bc7cc486699"}]: dispatch 2024-03-23T13:13:36.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:36 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "258432a8-3995-4878-928f-1bc7cc486699"}]': finished 2024-03-23T13:13:36.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:36 smithi007 ceph-mon[32184]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T13:13:36.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:36 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:36.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:36 smithi007 ceph-mon[32184]: from='client.? 172.21.15.130:0/622898459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:13:36.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:36 smithi130 ceph-mon[36225]: from='client.? 172.21.15.130:0/3230312749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "258432a8-3995-4878-928f-1bc7cc486699"}]: dispatch 2024-03-23T13:13:36.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:36 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "258432a8-3995-4878-928f-1bc7cc486699"}]: dispatch 2024-03-23T13:13:36.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:36 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "258432a8-3995-4878-928f-1bc7cc486699"}]': finished 2024-03-23T13:13:36.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:36 smithi130 ceph-mon[36225]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T13:13:36.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:36.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:36 smithi130 ceph-mon[36225]: from='client.? 172.21.15.130:0/622898459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:13:37.689 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:37 smithi130 ceph-mon[36225]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:13:37.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:37 smithi155 ceph-mon[36157]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:13:37.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:37 smithi007 ceph-mon[32184]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:13:39.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:39 smithi130 ceph-mon[36225]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-23T13:13:39.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:39 smithi155 ceph-mon[36157]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-23T13:13:39.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:39 smithi007 ceph-mon[32184]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-23T13:13:41.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:41 smithi130 ceph-mon[36225]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:41.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:41 smithi155 ceph-mon[36157]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:41 smithi007 ceph-mon[32184]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:43.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:43 smithi130 ceph-mon[36225]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:43.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T13:13:43.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:43.767 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:43 smithi130 ceph-mon[36225]: Deploying daemon osd.4 on smithi130 2024-03-23T13:13:43.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:43 smithi155 ceph-mon[36157]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:43.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T13:13:43.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:43.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:43 smithi155 ceph-mon[36157]: Deploying daemon osd.4 on smithi130 2024-03-23T13:13:43.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:43 smithi007 ceph-mon[32184]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:43.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T13:13:43.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:43.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:43 smithi007 ceph-mon[32184]: Deploying daemon osd.4 on smithi130 2024-03-23T13:13:45.760 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:45 smithi130 ceph-mon[36225]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:45.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:45 smithi155 ceph-mon[36157]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:45.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:45 smithi007 ceph-mon[32184]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:47.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:47 smithi130 ceph-mon[36225]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:47.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:47 smithi155 ceph-mon[36157]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:47.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:47 smithi007 ceph-mon[32184]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:49.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:49 smithi130 ceph-mon[36225]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:49.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:49 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:49.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:49 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:49.690 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:49 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:49.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:49 smithi155 ceph-mon[36157]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:49.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:49 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:49.783 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:49 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:49.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:49 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:49.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:49 smithi007 ceph-mon[32184]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:49.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:49 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:49.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:49 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:49.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:49 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:51.629 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:51 smithi130 ceph-mon[36225]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:51.630 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:51 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:51.630 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:51 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:51.630 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:51 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:51.630 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:51 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:51.630 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:51 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:51.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:51 smithi155 ceph-mon[36157]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:51.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:51 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:51.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:51 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:51.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:51 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:51.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:51 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:51.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:51 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:51.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:51 smithi007 ceph-mon[32184]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:51.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:51 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:51.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:51 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:51.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:51 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:51.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:51 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:51.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:51 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:51.998 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 4 on host 'smithi130' 2024-03-23T13:13:51.998 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.002+0000 7fe839ffb700 1 -- 172.21.15.130:0/158484421 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe83c02dc00 con 0x7fe82c0a2150 2024-03-23T13:13:52.000 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 -- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe82c0a2150 msgr2=0x7fe82c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:52.000 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe82c0a2150 0x7fe82c0a4610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe8280099d0 tx=0x7fe828008040 comp rx=0 tx=0).stop 2024-03-23T13:13:52.000 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 -- 172.21.15.130:0/158484421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c102220 msgr2=0x7fe83c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:52.000 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c102220 0x7fe83c1982e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe83000d980 tx=0x7fe83000dd40 comp rx=0 tx=0).stop 2024-03-23T13:13:52.000 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 -- 172.21.15.130:0/158484421 shutdown_connections 2024-03-23T13:13:52.000 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe82c0a2150 0x7fe82c0a4610 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:52.000 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe83c103dc0 0x7fe83c19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:52.000 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe83c102220 0x7fe83c1982e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:52.000 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 --2- 172.21.15.130:0/158484421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c103420 0x7fe83c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:52.000 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 -- 172.21.15.130:0/158484421 >> 172.21.15.130:0/158484421 conn(0x7fe83c0fd7e0 msgr2=0x7fe83c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:13:52.001 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 -- 172.21.15.130:0/158484421 shutdown_connections 2024-03-23T13:13:52.001 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:13:52.004+0000 7fe842f9e700 1 -- 172.21.15.130:0/158484421 wait complete. 2024-03-23T13:13:52.628 DEBUG:teuthology.orchestra.run.smithi130:osd.4> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.4.service 2024-03-23T13:13:52.630 INFO:tasks.cephadm:Deploying osd.5 on smithi155 with /dev/vg_nvme/lv_3... 2024-03-23T13:13:52.631 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T13:13:52.879 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:13:53.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:52 smithi155 ceph-mon[36157]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:53.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:52 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:53.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:52 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:53.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:52 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:53.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:52 smithi155 ceph-mon[36157]: from='osd.4 [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T13:13:53.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:52 smithi155 ceph-mon[36157]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T13:13:53.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:52 smithi007 ceph-mon[32184]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:53.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:52 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:53.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:52 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:53.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:52 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:53.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:52 smithi007 ceph-mon[32184]: from='osd.4 [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T13:13:53.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:52 smithi007 ceph-mon[32184]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T13:13:53.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:52 smithi130 ceph-mon[36225]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:53.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:52 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:53.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:52 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:53.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:52 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:13:53.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:52 smithi130 ceph-mon[36225]: from='osd.4 [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T13:13:53.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:52 smithi130 ceph-mon[36225]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T13:13:54.057 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:53 smithi130 ceph-mon[36225]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T13:13:54.057 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:53 smithi130 ceph-mon[36225]: from='osd.4 [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:13:54.057 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:53 smithi130 ceph-mon[36225]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T13:13:54.057 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:54.057 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:53 smithi130 ceph-mon[36225]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:13:54.057 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:13:53 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4[47820]: 2024-03-23T13:13:53.995+0000 7f119abb9700 -1 osd.4 0 waiting for initial osdmap 2024-03-23T13:13:54.058 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:13:54 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4[47820]: 2024-03-23T13:13:54.000+0000 7f11969ac700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:13:54.267 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:53 smithi155 ceph-mon[36157]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T13:13:54.267 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:53 smithi155 ceph-mon[36157]: from='osd.4 [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:13:54.267 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:53 smithi155 ceph-mon[36157]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T13:13:54.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:54.268 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:53 smithi155 ceph-mon[36157]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:13:54.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:53 smithi007 ceph-mon[32184]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T13:13:54.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:53 smithi007 ceph-mon[32184]: from='osd.4 [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:13:54.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:53 smithi007 ceph-mon[32184]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T13:13:54.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:54.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:53 smithi007 ceph-mon[32184]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:13:54.835 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T13:13:54.835 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T13:13:54.835 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-23T13:13:54.835 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-23T13:13:54.835 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121502 s, 863 MB/s 2024-03-23T13:13:54.835 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-23T13:13:55.101 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:55 smithi155 ceph-mon[36157]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:55.101 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:55 smithi155 ceph-mon[36157]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:13:55.101 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:55 smithi155 ceph-mon[36157]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T13:13:55.101 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:55.102 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:55.146 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:55 smithi130 ceph-mon[36225]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:55.146 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:55 smithi130 ceph-mon[36225]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:13:55.146 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:55 smithi130 ceph-mon[36225]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T13:13:55.146 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:55.146 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:55.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:55 smithi007 ceph-mon[32184]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:55.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:55 smithi007 ceph-mon[32184]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:13:55.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:55 smithi007 ceph-mon[32184]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T13:13:55.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:55.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:55.429 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-03-23T13:13:55.676 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:13:56.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:13:56.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:13:56.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: osd.4 [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] boot 2024-03-23T13:13:56.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T13:13:56.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:56.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi130 2024-03-23T13:13:56.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:56.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:56.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi130 to 8540M 2024-03-23T13:13:56.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:56.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:56.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:56 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:13:56.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:13:56.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: osd.4 [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] boot 2024-03-23T13:13:56.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T13:13:56.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:56.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi130 2024-03-23T13:13:56.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:56.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:56.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi130 to 8540M 2024-03-23T13:13:56.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:56.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:56.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:56 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:13:56.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:13:56.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: osd.4 [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] boot 2024-03-23T13:13:56.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T13:13:56.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T13:13:56.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi130 2024-03-23T13:13:56.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:56.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:13:56.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi130 to 8540M 2024-03-23T13:13:56.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:56.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:56.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:13:56.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:56 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:13:57.078 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:57 smithi155 ceph-mon[36157]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:57.079 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:57 smithi155 ceph-mon[36157]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T13:13:57.314 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.318+0000 7fbe40dd1700 1 -- 172.21.15.155:0/807034029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c101dc0 msgr2=0x7fbe3c109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:57.314 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.318+0000 7fbe40dd1700 1 --2- 172.21.15.155:0/807034029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c101dc0 0x7fbe3c109680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400b0a0 tx=0x7fbe3400b3b0 comp rx=0 tx=0).stop 2024-03-23T13:13:57.314 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.318+0000 7fbe40dd1700 1 -- 172.21.15.155:0/807034029 shutdown_connections 2024-03-23T13:13:57.314 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.318+0000 7fbe40dd1700 1 --2- 172.21.15.155:0/807034029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c101dc0 0x7fbe3c109680 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:57.314 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.318+0000 7fbe40dd1700 1 --2- 172.21.15.155:0/807034029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c1026e0 0x7fbe3c102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:57.314 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.318+0000 7fbe40dd1700 1 --2- 172.21.15.155:0/807034029 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101420 0x7fbe3c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:57.314 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.318+0000 7fbe40dd1700 1 -- 172.21.15.155:0/807034029 >> 172.21.15.155:0/807034029 conn(0x7fbe3c0fbe50 msgr2=0x7fbe3c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:13:57.314 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.318+0000 7fbe40dd1700 1 -- 172.21.15.155:0/807034029 shutdown_connections 2024-03-23T13:13:57.315 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.318+0000 7fbe40dd1700 1 -- 172.21.15.155:0/807034029 wait complete. 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.318+0000 7fbe40dd1700 1 Processor -- start 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe40dd1700 1 -- start start 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe40dd1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c101420 0x7fbe3c100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe40dd1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101dc0 0x7fbe3c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe40dd1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1026e0 0x7fbe3c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe40dd1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c19fd70 con 0x7fbe3c101420 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe40dd1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c19fee0 con 0x7fbe3c101dc0 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe40dd1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c1a00a0 con 0x7fbe3c1026e0 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe3ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1026e0 0x7fbe3c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe3ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1026e0 0x7fbe3c19f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:37910/0 (socket says 172.21.15.155:37910) 2024-03-23T13:13:57.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe3ad9d700 1 -- 172.21.15.155:0/4233267648 learned_addr learned my addr 172.21.15.155:0/4233267648 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe3a59c700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c101420 0x7fbe3c100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe39d9b700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101dc0 0x7fbe3c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe3ad9d700 1 -- 172.21.15.155:0/4233267648 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101dc0 msgr2=0x7fbe3c19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe3ad9d700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101dc0 0x7fbe3c19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe3ad9d700 1 -- 172.21.15.155:0/4233267648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c101420 msgr2=0x7fbe3c100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe3ad9d700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c101420 0x7fbe3c100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe3ad9d700 1 -- 172.21.15.155:0/4233267648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe34009d00 con 0x7fbe3c1026e0 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.319+0000 7fbe3ad9d700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1026e0 0x7fbe3c19f5d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400b070 tx=0x7fbe340093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.320+0000 7fbe2b7fe700 1 -- 172.21.15.155:0/4233267648 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3400e040 con 0x7fbe3c1026e0 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.320+0000 7fbe2b7fe700 1 -- 172.21.15.155:0/4233267648 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe3401d960 con 0x7fbe3c1026e0 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.320+0000 7fbe2b7fe700 1 -- 172.21.15.155:0/4233267648 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3401ba80 con 0x7fbe3c1026e0 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.320+0000 7fbe40dd1700 1 -- 172.21.15.155:0/4233267648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3c1a0320 con 0x7fbe3c1026e0 2024-03-23T13:13:57.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.320+0000 7fbe40dd1700 1 -- 172.21.15.155:0/4233267648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3c1a0740 con 0x7fbe3c1026e0 2024-03-23T13:13:57.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.321+0000 7fbe40dd1700 1 -- 172.21.15.155:0/4233267648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3c108520 con 0x7fbe3c1026e0 2024-03-23T13:13:57.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.322+0000 7fbe2b7fe700 1 -- 172.21.15.155:0/4233267648 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fbe3401dad0 con 0x7fbe3c1026e0 2024-03-23T13:13:57.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.322+0000 7fbe2b7fe700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe240a2150 0x7fbe240a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:13:57.319 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.322+0000 7fbe2b7fe700 1 -- 172.21.15.155:0/4233267648 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fbe340ce9e0 con 0x7fbe3c1026e0 2024-03-23T13:13:57.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.322+0000 7fbe3a59c700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe240a2150 0x7fbe240a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:13:57.320 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.323+0000 7fbe3a59c700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe240a2150 0x7fbe240a4610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c06a700 tx=0x7fbe2c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:13:57.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:57 smithi007 ceph-mon[32184]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:57.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:57 smithi007 ceph-mon[32184]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T13:13:57.323 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.327+0000 7fbe2b7fe700 1 -- 172.21.15.155:0/4233267648 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fbe34098ab0 con 0x7fbe3c1026e0 2024-03-23T13:13:57.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:57 smithi130 ceph-mon[36225]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T13:13:57.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:57 smithi130 ceph-mon[36225]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T13:13:57.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:13:57.528+0000 7fbe40dd1700 1 -- 172.21.15.155:0/4233267648 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbe3c02dc00 con 0x7fbe240a2150 2024-03-23T13:13:58.271 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:58 smithi155 ceph-mon[36157]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:13:58.271 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:58 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:13:58.272 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:58 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:13:58.272 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:58 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:58.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:58 smithi007 ceph-mon[32184]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:13:58.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:13:58.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:13:58.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:58.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:58 smithi130 ceph-mon[36225]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:13:58.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:58 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:13:58.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:58 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:13:58.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:58 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:13:59.275 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:13:59 smithi155 ceph-mon[36157]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:13:59.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:13:59 smithi007 ceph-mon[32184]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:13:59.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:13:59 smithi130 ceph-mon[36225]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:01.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:01 smithi155 ceph-mon[36157]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:01.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:01 smithi155 ceph-mon[36157]: from='client.? 172.21.15.155:0/3123360193' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f115a6d4-3043-48d6-a29e-b367c62dd418"}]: dispatch 2024-03-23T13:14:01.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:01 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f115a6d4-3043-48d6-a29e-b367c62dd418"}]: dispatch 2024-03-23T13:14:01.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:01 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f115a6d4-3043-48d6-a29e-b367c62dd418"}]': finished 2024-03-23T13:14:01.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:01 smithi155 ceph-mon[36157]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T13:14:01.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:01 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:01.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:01 smithi155 ceph-mon[36157]: from='client.? 172.21.15.155:0/948358632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:14:01.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:01 smithi007 ceph-mon[32184]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:01.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:01 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/3123360193' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f115a6d4-3043-48d6-a29e-b367c62dd418"}]: dispatch 2024-03-23T13:14:01.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:01 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f115a6d4-3043-48d6-a29e-b367c62dd418"}]: dispatch 2024-03-23T13:14:01.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:01 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f115a6d4-3043-48d6-a29e-b367c62dd418"}]': finished 2024-03-23T13:14:01.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:01 smithi007 ceph-mon[32184]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T13:14:01.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:01 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:01.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:01 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/948358632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:14:01.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:01 smithi130 ceph-mon[36225]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:01.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:01 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/3123360193' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f115a6d4-3043-48d6-a29e-b367c62dd418"}]: dispatch 2024-03-23T13:14:01.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:01 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f115a6d4-3043-48d6-a29e-b367c62dd418"}]: dispatch 2024-03-23T13:14:01.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:01 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f115a6d4-3043-48d6-a29e-b367c62dd418"}]': finished 2024-03-23T13:14:01.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:01 smithi130 ceph-mon[36225]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T13:14:01.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:01 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:01.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:01 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/948358632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:14:03.283 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:03 smithi155 ceph-mon[36157]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:03.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:03 smithi007 ceph-mon[32184]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:03.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:03 smithi130 ceph-mon[36225]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:05.280 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:05 smithi155 ceph-mon[36157]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:05.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:05 smithi007 ceph-mon[32184]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:05.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:05 smithi130 ceph-mon[36225]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:07.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:07 smithi155 ceph-mon[36157]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:07.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:07 smithi007 ceph-mon[32184]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:07.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:07 smithi130 ceph-mon[36225]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:08.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:08 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T13:14:08.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:08 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:08.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:08 smithi155 ceph-mon[36157]: Deploying daemon osd.5 on smithi155 2024-03-23T13:14:08.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:08 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T13:14:08.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:08 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:08.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:08 smithi007 ceph-mon[32184]: Deploying daemon osd.5 on smithi155 2024-03-23T13:14:08.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:08 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T13:14:08.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:08 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:08.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:08 smithi130 ceph-mon[36225]: Deploying daemon osd.5 on smithi155 2024-03-23T13:14:09.186 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:09 smithi155 ceph-mon[36157]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:09.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:09 smithi007 ceph-mon[32184]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:09.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:09 smithi130 ceph-mon[36225]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:11.186 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:11 smithi155 ceph-mon[36157]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:11.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:11 smithi007 ceph-mon[32184]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:11.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:11 smithi130 ceph-mon[36225]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:13.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:13 smithi155 ceph-mon[36157]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:13.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:13 smithi007 ceph-mon[32184]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:13.440 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:13 smithi130 ceph-mon[36225]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:14.131 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:14.132 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:14.132 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:14.442 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:14.442 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:14.442 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:15.284 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:15 smithi155 ceph-mon[36157]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:15.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:15 smithi130 ceph-mon[36225]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:15.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:15 smithi007 ceph-mon[32184]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:16.644 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 5 on host 'smithi155' 2024-03-23T13:14:16.644 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.647+0000 7fbe2b7fe700 1 -- 172.21.15.155:0/4233267648 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbe3c02dc00 con 0x7fbe240a2150 2024-03-23T13:14:16.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 -- 172.21.15.155:0/4233267648 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe240a2150 msgr2=0x7fbe240a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:14:16.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe240a2150 0x7fbe240a4610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c06a700 tx=0x7fbe2c007400 comp rx=0 tx=0).stop 2024-03-23T13:14:16.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 -- 172.21.15.155:0/4233267648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1026e0 msgr2=0x7fbe3c19f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:14:16.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1026e0 0x7fbe3c19f5d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400b070 tx=0x7fbe340093f0 comp rx=0 tx=0).stop 2024-03-23T13:14:16.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 -- 172.21.15.155:0/4233267648 shutdown_connections 2024-03-23T13:14:16.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe240a2150 0x7fbe240a4610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:16.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1026e0 0x7fbe3c19f5d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:16.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c101420 0x7fbe3c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:16.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 --2- 172.21.15.155:0/4233267648 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101dc0 0x7fbe3c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:16.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 -- 172.21.15.155:0/4233267648 >> 172.21.15.155:0/4233267648 conn(0x7fbe3c0fbe50 msgr2=0x7fbe3c105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:14:16.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 -- 172.21.15.155:0/4233267648 shutdown_connections 2024-03-23T13:14:16.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:14:16.651+0000 7fbe40dd1700 1 -- 172.21.15.155:0/4233267648 wait complete. 2024-03-23T13:14:17.310 DEBUG:teuthology.orchestra.run.smithi155:osd.5> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.5.service 2024-03-23T13:14:17.312 INFO:tasks.cephadm:Deploying osd.6 on smithi007 with /dev/vg_nvme/lv_2... 2024-03-23T13:14:17.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T13:14:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:14:17.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:17 smithi007 ceph-mon[32184]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:17.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:17.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:17.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:17 smithi007 ceph-mon[32184]: from='osd.5 [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T13:14:17.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:17 smithi007 ceph-mon[32184]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T13:14:17.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:17.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:17.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:17 smithi130 ceph-mon[36225]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:17.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:17.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:17.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:17 smithi130 ceph-mon[36225]: from='osd.5 [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T13:14:17.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:17 smithi130 ceph-mon[36225]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T13:14:17.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:17.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:17.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:17 smithi155 ceph-mon[36157]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:17.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:17.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:17.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:17 smithi155 ceph-mon[36157]: from='osd.5 [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T13:14:18.000 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:17 smithi155 ceph-mon[36157]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T13:14:18.000 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:18.000 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:18.938 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:18 smithi007 ceph-mon[32184]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T13:14:18.939 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:18 smithi007 ceph-mon[32184]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T13:14:18.939 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:18 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:18.939 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:18 smithi007 ceph-mon[32184]: from='osd.5 [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:14:18.939 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:18 smithi007 ceph-mon[32184]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:14:18.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:18 smithi130 ceph-mon[36225]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T13:14:18.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:18 smithi130 ceph-mon[36225]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T13:14:18.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:18 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:18.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:18 smithi130 ceph-mon[36225]: from='osd.5 [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:14:18.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:18 smithi130 ceph-mon[36225]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:14:19.021 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:18 smithi155 ceph-mon[36157]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T13:14:19.021 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:18 smithi155 ceph-mon[36157]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T13:14:19.022 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:18 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:19.022 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:18 smithi155 ceph-mon[36157]: from='osd.5 [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:14:19.022 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:18 smithi155 ceph-mon[36157]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:14:19.022 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:14:18 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:14:18.649+0000 7fdc8e303700 -1 osd.5 0 waiting for initial osdmap 2024-03-23T13:14:19.022 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:14:18 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:14:18.655+0000 7fdc8a0f6700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:14:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T13:14:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T13:14:19.632 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-23T13:14:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-23T13:14:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00848476 s, 1.2 GB/s 2024-03-23T13:14:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-23T13:14:19.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:19 smithi007 ceph-mon[32184]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:19.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:19 smithi007 ceph-mon[32184]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:14:19.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:19 smithi007 ceph-mon[32184]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T13:14:19.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:19.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:19 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:19.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:19 smithi130 ceph-mon[36225]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:19.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:19 smithi130 ceph-mon[36225]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:14:19.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:19 smithi130 ceph-mon[36225]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T13:14:19.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:19.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:19 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:20.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:19 smithi155 ceph-mon[36157]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:20.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:19 smithi155 ceph-mon[36157]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:14:20.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:19 smithi155 ceph-mon[36157]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T13:14:20.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:20.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:19 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:20.283 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_2 2024-03-23T13:14:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:14:20.720 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:20 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:14:20.721 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:20 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:14:20.721 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:20 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:20.721 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:20 smithi155 ceph-mon[36157]: osd.5 [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] boot 2024-03-23T13:14:20.721 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:20 smithi155 ceph-mon[36157]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T13:14:20.721 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:20 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:20.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:20 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:14:20.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:20 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:14:20.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:20 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:20.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:20 smithi130 ceph-mon[36225]: osd.5 [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] boot 2024-03-23T13:14:20.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:20 smithi130 ceph-mon[36225]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T13:14:20.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:20 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:20.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:20 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:14:20.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:20 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:14:20.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:20 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:20.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:20 smithi007 ceph-mon[32184]: osd.5 [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] boot 2024-03-23T13:14:20.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:20 smithi007 ceph-mon[32184]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T13:14:20.977 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:20 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T13:14:21.864 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:21.864 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T13:14:21.864 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi155 2024-03-23T13:14:21.864 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:21.864 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi155 to 8540M 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:21.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:21.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi155 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi155 to 8540M 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:21.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:22.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-23T13:14:22.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T13:14:22.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi155 2024-03-23T13:14:22.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:22.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi155 to 8540M 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:22.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 -- 172.21.15.7:0/1931300440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c103430 msgr2=0x7f205c103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:14:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 --2- 172.21.15.7:0/1931300440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c103430 0x7f205c103890 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f20440099c0 tx=0x7f2044009cd0 comp rx=0 tx=0).stop 2024-03-23T13:14:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 -- 172.21.15.7:0/1931300440 shutdown_connections 2024-03-23T13:14:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 --2- 172.21.15.7:0/1931300440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f205c103dd0 0x7f205c10a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 --2- 172.21.15.7:0/1931300440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f205c102230 0x7f205c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 --2- 172.21.15.7:0/1931300440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c103430 0x7f205c103890 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 -- 172.21.15.7:0/1931300440 >> 172.21.15.7:0/1931300440 conn(0x7f205c0fd810 msgr2=0x7f205c0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:14:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 -- 172.21.15.7:0/1931300440 shutdown_connections 2024-03-23T13:14:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 -- 172.21.15.7:0/1931300440 wait complete. 2024-03-23T13:14:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 Processor -- start 2024-03-23T13:14:22.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.097+0000 7f20624c1700 1 -- start start 2024-03-23T13:14:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20624c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102230 0x7f205c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:14:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20624c1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f205c103430 0x7f205c0745f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:14:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20624c1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f205c103dd0 0x7f205c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:14:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20624c1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f205c0752b0 con 0x7f205c102230 2024-03-23T13:14:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20624c1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f205c075420 con 0x7f205c103dd0 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20624c1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f205c1a21d0 con 0x7f205c103430 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f205bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102230 0x7f205c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f205bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102230 0x7f205c075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48028/0 (socket says 172.21.15.7:48028) 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f205bfff700 1 -- 172.21.15.7:0/2002254016 learned_addr learned my addr 172.21.15.7:0/2002254016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f205b7fe700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f205c103430 0x7f205c0745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f2060a5e700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f205c103dd0 0x7f205c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f205bfff700 1 -- 172.21.15.7:0/2002254016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f205c103430 msgr2=0x7f205c0745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f205bfff700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f205c103430 0x7f205c0745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f205bfff700 1 -- 172.21.15.7:0/2002254016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f205c103dd0 msgr2=0x7f205c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f205bfff700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f205c103dd0 0x7f205c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f205bfff700 1 -- 172.21.15.7:0/2002254016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20440095f0 con 0x7f205c102230 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f2060a5e700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f205c103dd0 0x7f205c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f205bfff700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102230 0x7f205c075fa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f204c00c990 tx=0x7f204c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:14:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20597fa700 1 -- 172.21.15.7:0/2002254016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c007b00 con 0x7f205c102230 2024-03-23T13:14:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20597fa700 1 -- 172.21.15.7:0/2002254016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f204c007c60 con 0x7f205c102230 2024-03-23T13:14:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20624c1700 1 -- 172.21.15.7:0/2002254016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c1a24b0 con 0x7f205c102230 2024-03-23T13:14:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20597fa700 1 -- 172.21.15.7:0/2002254016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c018660 con 0x7f205c102230 2024-03-23T13:14:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.098+0000 7f20624c1700 1 -- 172.21.15.7:0/2002254016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c1a29b0 con 0x7f205c102230 2024-03-23T13:14:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.099+0000 7f20624c1700 1 -- 172.21.15.7:0/2002254016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c1095f0 con 0x7f205c102230 2024-03-23T13:14:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.101+0000 7f20597fa700 1 -- 172.21.15.7:0/2002254016 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f204c004020 con 0x7f205c102230 2024-03-23T13:14:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.102+0000 7f20597fa700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f20480a2150 0x7f20480a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:14:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.102+0000 7f205b7fe700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f20480a2150 0x7f20480a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:14:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.102+0000 7f20597fa700 1 -- 172.21.15.7:0/2002254016 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f204c0cd0d0 con 0x7f205c102230 2024-03-23T13:14:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.102+0000 7f205b7fe700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f20480a2150 0x7f20480a4610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2044000c00 tx=0x7f2044005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:14:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.106+0000 7f20597fa700 1 -- 172.21.15.7:0/2002254016 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f204c096ea0 con 0x7f205c102230 2024-03-23T13:14:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:22.317+0000 7f20624c1700 1 -- 172.21.15.7:0/2002254016 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f205c02dc00 con 0x7f20480a2150 2024-03-23T13:14:22.926 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:22 smithi007 ceph-mon[32184]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T13:14:22.927 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:22 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:14:22.927 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:22 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:14:22.927 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:22 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:22.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:22 smithi130 ceph-mon[36225]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T13:14:22.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:22 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:14:22.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:22 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:14:22.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:22 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:23.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:22 smithi155 ceph-mon[36157]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T13:14:23.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:22 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:14:23.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:22 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:14:23.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:22 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:23.930 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:23 smithi007 ceph-mon[32184]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:23.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:23 smithi007 ceph-mon[32184]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:14:23.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:23 smithi007 ceph-mon[32184]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T13:14:23.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:23 smithi130 ceph-mon[36225]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:23.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:23 smithi130 ceph-mon[36225]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:14:23.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:23 smithi130 ceph-mon[36225]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T13:14:24.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:23 smithi155 ceph-mon[36157]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:24.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:23 smithi155 ceph-mon[36157]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:14:24.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:23 smithi155 ceph-mon[36157]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T13:14:24.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:24 smithi130 ceph-mon[36225]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T13:14:25.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:24 smithi155 ceph-mon[36157]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T13:14:25.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:24 smithi007 ceph-mon[32184]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T13:14:25.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:25 smithi130 ceph-mon[36225]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:14:25.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:25 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3605502377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16353022-32db-487a-99e8-d8fe28fdaef0"}]: dispatch 2024-03-23T13:14:25.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:25 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3605502377' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16353022-32db-487a-99e8-d8fe28fdaef0"}]': finished 2024-03-23T13:14:25.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:25 smithi130 ceph-mon[36225]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T13:14:25.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:25 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:25.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:25 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/4266987658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:14:26.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:25 smithi155 ceph-mon[36157]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:14:26.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:25 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3605502377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16353022-32db-487a-99e8-d8fe28fdaef0"}]: dispatch 2024-03-23T13:14:26.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:25 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3605502377' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16353022-32db-487a-99e8-d8fe28fdaef0"}]': finished 2024-03-23T13:14:26.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:25 smithi155 ceph-mon[36157]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T13:14:26.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:25 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:26.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:25 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/4266987658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:14:26.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:25 smithi007 ceph-mon[32184]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:14:26.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:25 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3605502377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16353022-32db-487a-99e8-d8fe28fdaef0"}]: dispatch 2024-03-23T13:14:26.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:25 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3605502377' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16353022-32db-487a-99e8-d8fe28fdaef0"}]': finished 2024-03-23T13:14:26.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:25 smithi007 ceph-mon[32184]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T13:14:26.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:25 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:26.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:25 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4266987658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:14:27.933 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:27 smithi155 ceph-mon[36157]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-23T13:14:27.940 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:27 smithi130 ceph-mon[36225]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-23T13:14:28.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:27 smithi007 ceph-mon[32184]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-23T13:14:29.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:29 smithi130 ceph-mon[36225]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T13:14:30.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:29 smithi155 ceph-mon[36157]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T13:14:30.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:29 smithi007 ceph-mon[32184]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T13:14:31.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:31 smithi130 ceph-mon[36225]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-23T13:14:32.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:31 smithi155 ceph-mon[36157]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-23T13:14:32.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:31 smithi007 ceph-mon[32184]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-23T13:14:33.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T13:14:33.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:33.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T13:14:33.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:33.191 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T13:14:33.191 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:34.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:33 smithi155 ceph-mon[36157]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:34.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:33 smithi155 ceph-mon[36157]: Deploying daemon osd.6 on smithi007 2024-03-23T13:14:34.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:33 smithi007 ceph-mon[32184]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:34.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:33 smithi007 ceph-mon[32184]: Deploying daemon osd.6 on smithi007 2024-03-23T13:14:34.094 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:33 smithi130 ceph-mon[36225]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:34.094 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:33 smithi130 ceph-mon[36225]: Deploying daemon osd.6 on smithi007 2024-03-23T13:14:36.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:35 smithi155 ceph-mon[36157]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:36.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:35 smithi007 ceph-mon[32184]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:36.190 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:35 smithi130 ceph-mon[36225]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:37.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:37 smithi007 ceph-mon[32184]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:38.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:37 smithi155 ceph-mon[36157]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:38.191 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:37 smithi130 ceph-mon[36225]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:39.691 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:39 smithi130 ceph-mon[36225]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:39.691 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:39.691 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:39.691 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:39.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:39 smithi155 ceph-mon[36157]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:39.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:39.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:39.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:39.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:39 smithi007 ceph-mon[32184]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:39.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:39.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:39.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:41.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:41 smithi155 ceph-mon[36157]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:41.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:41.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:41.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:41.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:41.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:41.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:41 smithi007 ceph-mon[32184]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:41.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:41.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:41.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:41.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:41.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:41.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:41 smithi130 ceph-mon[36225]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:41.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:41 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:41.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:41 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:41.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:41 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:41.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:41 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:41.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:41 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:42.334 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 6 on host 'smithi007' 2024-03-23T13:14:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.338+0000 7f20597fa700 1 -- 172.21.15.7:0/2002254016 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f205c02dc00 con 0x7f20480a2150 2024-03-23T13:14:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 -- 172.21.15.7:0/2002254016 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f20480a2150 msgr2=0x7f20480a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:14:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f20480a2150 0x7f20480a4610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2044000c00 tx=0x7f2044005fd0 comp rx=0 tx=0).stop 2024-03-23T13:14:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 -- 172.21.15.7:0/2002254016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102230 msgr2=0x7f205c075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:14:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102230 0x7f205c075fa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f204c00c990 tx=0x7f204c00cd50 comp rx=0 tx=0).stop 2024-03-23T13:14:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 -- 172.21.15.7:0/2002254016 shutdown_connections 2024-03-23T13:14:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f20480a2150 0x7f20480a4610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f205c103dd0 0x7f205c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102230 0x7f205c075fa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 --2- 172.21.15.7:0/2002254016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f205c103430 0x7f205c0745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 -- 172.21.15.7:0/2002254016 >> 172.21.15.7:0/2002254016 conn(0x7f205c0fd810 msgr2=0x7f205c106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:14:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 -- 172.21.15.7:0/2002254016 shutdown_connections 2024-03-23T13:14:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:14:42.340+0000 7f20624c1700 1 -- 172.21.15.7:0/2002254016 wait complete. 2024-03-23T13:14:43.025 DEBUG:teuthology.orchestra.run.smithi007:osd.6> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.6.service 2024-03-23T13:14:43.027 INFO:tasks.cephadm:Deploying osd.7 on smithi130 with /dev/vg_nvme/lv_2... 2024-03-23T13:14:43.027 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T13:14:43.270 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:14:43.391 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:43 smithi130 ceph-mon[36225]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:43.391 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:43.392 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:43.392 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:43.392 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:43 smithi130 ceph-mon[36225]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T13:14:43.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:43 smithi007 ceph-mon[32184]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:43.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:43.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:43.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:43.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:43 smithi007 ceph-mon[32184]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T13:14:43.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:43 smithi155 ceph-mon[36157]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:43.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:43.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:43.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:14:43.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:43 smithi155 ceph-mon[36157]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T13:14:44.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:44 smithi130 ceph-mon[36225]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T13:14:44.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:44 smithi130 ceph-mon[36225]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T13:14:44.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:44 smithi130 ceph-mon[36225]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:14:44.441 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:44 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:44.528 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:44 smithi007 ceph-mon[32184]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T13:14:44.529 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:44 smithi007 ceph-mon[32184]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T13:14:44.529 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:44 smithi007 ceph-mon[32184]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:14:44.529 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:44 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:44.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:44 smithi155 ceph-mon[36157]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T13:14:44.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:44 smithi155 ceph-mon[36157]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T13:14:44.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:44 smithi155 ceph-mon[36157]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:14:44.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:44 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:44.824 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:14:44 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:14:44.762+0000 7f23a31c8700 -1 osd.6 0 waiting for initial osdmap 2024-03-23T13:14:44.824 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:14:44 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:14:44.771+0000 7f239afb3700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:14:45.422 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T13:14:45.423 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T13:14:45.423 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-03-23T13:14:45.423 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-03-23T13:14:45.423 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138163 s, 759 MB/s 2024-03-23T13:14:45.423 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-03-23T13:14:45.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:45 smithi130 ceph-mon[36225]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:45.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:45 smithi130 ceph-mon[36225]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:14:45.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:45 smithi130 ceph-mon[36225]: osdmap e45: 7 total, 6 up, 7 in 2024-03-23T13:14:45.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:45 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:45.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:45 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:45.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:45 smithi130 ceph-mon[36225]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' 2024-03-23T13:14:45.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:45 smithi007 ceph-mon[32184]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:45.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:45 smithi007 ceph-mon[32184]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:14:45.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:45 smithi007 ceph-mon[32184]: osdmap e45: 7 total, 6 up, 7 in 2024-03-23T13:14:45.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:45 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:45.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:45 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:45.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:45 smithi007 ceph-mon[32184]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' 2024-03-23T13:14:45.686 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:45 smithi155 ceph-mon[36157]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:45.686 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:45 smithi155 ceph-mon[36157]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:14:45.686 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:45 smithi155 ceph-mon[36157]: osdmap e45: 7 total, 6 up, 7 in 2024-03-23T13:14:45.686 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:45 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:45.686 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:45 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:45.686 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:45 smithi155 ceph-mon[36157]: from='osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524]' entity='osd.6' 2024-03-23T13:14:46.061 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi130:vg_nvme/lv_2 2024-03-23T13:14:46.306 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] boot 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: osdmap e46: 7 total, 7 up, 7 in 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:46.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:46.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:46.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:46 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.582 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:14:46.582 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:14:46.582 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] boot 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: osdmap e46: 7 total, 7 up, 7 in 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:46.583 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:46 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: osd.6 [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] boot 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: osdmap e46: 7 total, 7 up, 7 in 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:46.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:46.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:14:46.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:46 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:14:47.869 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.870+0000 7f95f9319700 1 -- 172.21.15.130:0/2348654146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95f4100ab0 msgr2=0x7f95f4102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:14:47.869 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.870+0000 7f95f9319700 1 --2- 172.21.15.130:0/2348654146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95f4100ab0 0x7f95f4102ee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009990 tx=0x7f95e4009ca0 comp rx=0 tx=0).stop 2024-03-23T13:14:47.869 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.870+0000 7f95f9319700 1 -- 172.21.15.130:0/2348654146 shutdown_connections 2024-03-23T13:14:47.869 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.870+0000 7f95f9319700 1 --2- 172.21.15.130:0/2348654146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4105df0 0x7f95f4106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:47.869 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.870+0000 7f95f9319700 1 --2- 172.21.15.130:0/2348654146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95f4100ab0 0x7f95f4102ee0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:47.869 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.870+0000 7f95f9319700 1 --2- 172.21.15.130:0/2348654146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95f4103420 0x7f95f41058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:47.869 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.870+0000 7f95f9319700 1 -- 172.21.15.130:0/2348654146 >> 172.21.15.130:0/2348654146 conn(0x7f95f40fc6d0 msgr2=0x7f95f40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:14:47.869 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.870+0000 7f95f9319700 1 -- 172.21.15.130:0/2348654146 shutdown_connections 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.870+0000 7f95f9319700 1 -- 172.21.15.130:0/2348654146 wait complete. 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f9319700 1 Processor -- start 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f9319700 1 -- start start 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f9319700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4100ab0 0x7f95f419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f9319700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95f4103420 0x7f95f419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f9319700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95f4105df0 0x7f95f41a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f9319700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41a63d0 con 0x7f95f4100ab0 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f9319700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41a6510 con 0x7f95f4103420 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f9319700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41a66a0 con 0x7f95f4105df0 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f27fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95f4103420 0x7f95f419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f27fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95f4103420 0x7f95f419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:34362/0 (socket says 172.21.15.130:34362) 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f27fc700 1 -- 172.21.15.130:0/910089301 learned_addr learned my addr 172.21.15.130:0/910089301 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-03-23T13:14:47.870 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f2ffd700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4100ab0 0x7f95f419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.871+0000 7f95f37fe700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95f4105df0 0x7f95f41a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95f27fc700 1 -- 172.21.15.130:0/910089301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95f4105df0 msgr2=0x7f95f41a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95f27fc700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95f4105df0 0x7f95f41a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95f27fc700 1 -- 172.21.15.130:0/910089301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4100ab0 msgr2=0x7f95f419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95f27fc700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4100ab0 0x7f95f419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95f27fc700 1 -- 172.21.15.130:0/910089301 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e40095f0 con 0x7f95f4103420 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95f27fc700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95f4103420 0x7f95f419edd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f95e800b530 tx=0x7f95e800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95dbfff700 1 -- 172.21.15.130:0/910089301 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8011830 con 0x7f95f4103420 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95f9319700 1 -- 172.21.15.130:0/910089301 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f41a6980 con 0x7f95f4103420 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95dbfff700 1 -- 172.21.15.130:0/910089301 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e8011e70 con 0x7f95f4103420 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95dbfff700 1 -- 172.21.15.130:0/910089301 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e800f2e0 con 0x7f95f4103420 2024-03-23T13:14:47.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.872+0000 7f95f9319700 1 -- 172.21.15.130:0/910089301 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f41a6e80 con 0x7f95f4103420 2024-03-23T13:14:47.872 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.873+0000 7f95f9319700 1 -- 172.21.15.130:0/910089301 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f40fe250 con 0x7f95f4103420 2024-03-23T13:14:47.872 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.874+0000 7f95dbfff700 1 -- 172.21.15.130:0/910089301 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f95e8010610 con 0x7f95f4103420 2024-03-23T13:14:47.872 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.875+0000 7f95dbfff700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f95dc0a2150 0x7f95dc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:14:47.872 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.875+0000 7f95dbfff700 1 -- 172.21.15.130:0/910089301 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f95e80ccda0 con 0x7f95f4103420 2024-03-23T13:14:47.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.875+0000 7f95f2ffd700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f95dc0a2150 0x7f95dc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:14:47.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.875+0000 7f95f2ffd700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f95dc0a2150 0x7f95dc0a4610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f95e400b5c0 tx=0x7f95e40038a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:14:47.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:47.878+0000 7f95dbfff700 1 -- 172.21.15.130:0/910089301 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f95e8096a80 con 0x7f95f4103420 2024-03-23T13:14:47.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:47 smithi130 ceph-mon[36225]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:47.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:47 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi007 2024-03-23T13:14:47.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:47 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi007 to 5697M 2024-03-23T13:14:47.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:47 smithi130 ceph-mon[36225]: osdmap e47: 7 total, 7 up, 7 in 2024-03-23T13:14:48.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:47 smithi155 ceph-mon[36157]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:48.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:47 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi007 2024-03-23T13:14:48.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:47 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi007 to 5697M 2024-03-23T13:14:48.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:47 smithi155 ceph-mon[36157]: osdmap e47: 7 total, 7 up, 7 in 2024-03-23T13:14:48.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:47 smithi007 ceph-mon[32184]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-23T13:14:48.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:47 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi007 2024-03-23T13:14:48.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:47 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi007 to 5697M 2024-03-23T13:14:48.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:47 smithi007 ceph-mon[32184]: osdmap e47: 7 total, 7 up, 7 in 2024-03-23T13:14:48.078 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:14:48.081+0000 7f95f9319700 1 -- 172.21.15.130:0/910089301 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f95f402dc30 con 0x7f95dc0a2150 2024-03-23T13:14:48.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:48 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:14:48.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:48 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:14:48.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:48 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:49.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:48 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:14:49.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:48 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:14:49.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:48 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:49.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:48 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:14:49.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:48 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:14:49.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:48 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:50.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:49 smithi155 ceph-mon[36157]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:50.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:49 smithi155 ceph-mon[36157]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:14:50.052 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:49 smithi130 ceph-mon[36225]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:50.052 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:49 smithi130 ceph-mon[36225]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:14:50.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:49 smithi007 ceph-mon[32184]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:50.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:49 smithi007 ceph-mon[32184]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:14:52.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:51 smithi155 ceph-mon[36157]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:52.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:51 smithi155 ceph-mon[36157]: from='client.? 172.21.15.130:0/191519955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b2652e-e404-4893-aff0-5049956327ad"}]: dispatch 2024-03-23T13:14:52.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:51 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b2652e-e404-4893-aff0-5049956327ad"}]: dispatch 2024-03-23T13:14:52.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:51 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3b2652e-e404-4893-aff0-5049956327ad"}]': finished 2024-03-23T13:14:52.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:51 smithi155 ceph-mon[36157]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T13:14:52.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:51 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:14:52.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:51 smithi155 ceph-mon[36157]: from='client.? 172.21.15.130:0/3755454685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:14:52.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:51 smithi007 ceph-mon[32184]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:52.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:51 smithi007 ceph-mon[32184]: from='client.? 172.21.15.130:0/191519955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b2652e-e404-4893-aff0-5049956327ad"}]: dispatch 2024-03-23T13:14:52.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:51 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b2652e-e404-4893-aff0-5049956327ad"}]: dispatch 2024-03-23T13:14:52.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:51 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3b2652e-e404-4893-aff0-5049956327ad"}]': finished 2024-03-23T13:14:52.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:51 smithi007 ceph-mon[32184]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T13:14:52.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:51 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:14:52.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:51 smithi007 ceph-mon[32184]: from='client.? 172.21.15.130:0/3755454685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:14:52.191 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:51 smithi130 ceph-mon[36225]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:52.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:51 smithi130 ceph-mon[36225]: from='client.? 172.21.15.130:0/191519955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b2652e-e404-4893-aff0-5049956327ad"}]: dispatch 2024-03-23T13:14:52.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:51 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b2652e-e404-4893-aff0-5049956327ad"}]: dispatch 2024-03-23T13:14:52.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:51 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3b2652e-e404-4893-aff0-5049956327ad"}]': finished 2024-03-23T13:14:52.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:51 smithi130 ceph-mon[36225]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T13:14:52.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:51 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:14:52.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:51 smithi130 ceph-mon[36225]: from='client.? 172.21.15.130:0/3755454685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:14:54.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:53 smithi155 ceph-mon[36157]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:54.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:53 smithi007 ceph-mon[32184]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:54.191 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:53 smithi130 ceph-mon[36225]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:56.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:55 smithi155 ceph-mon[36157]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:56.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:55 smithi007 ceph-mon[32184]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:56.092 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:55 smithi130 ceph-mon[36225]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:58.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:57 smithi155 ceph-mon[36157]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:58.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:57 smithi007 ceph-mon[32184]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:58.129 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:57 smithi130 ceph-mon[36225]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:14:59.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T13:14:59.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:58 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:59.090 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:58 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T13:14:59.091 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:58 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:14:59.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:58 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T13:14:59.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:58 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:00.053 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:59 smithi130 ceph-mon[36225]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:00.053 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:14:59 smithi130 ceph-mon[36225]: Deploying daemon osd.7 on smithi130 2024-03-23T13:15:00.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:59 smithi007 ceph-mon[32184]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:00.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:14:59 smithi007 ceph-mon[32184]: Deploying daemon osd.7 on smithi130 2024-03-23T13:15:00.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:59 smithi155 ceph-mon[36157]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:00.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:14:59 smithi155 ceph-mon[36157]: Deploying daemon osd.7 on smithi130 2024-03-23T13:15:02.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:01 smithi007 ceph-mon[32184]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:02.081 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:01 smithi130 ceph-mon[36225]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:02.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:01 smithi155 ceph-mon[36157]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:04.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:03 smithi007 ceph-mon[32184]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:04.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:04.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:04.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:04.155 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:03 smithi130 ceph-mon[36225]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:04.155 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:04.155 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:04.155 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:04.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:03 smithi155 ceph-mon[36157]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:04.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:04.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:04.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:05.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:05 smithi130 ceph-mon[36225]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:06.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:05 smithi007 ceph-mon[32184]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:06.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:05 smithi155 ceph-mon[36157]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:07.697 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.699+0000 7f95dbfff700 1 -- 172.21.15.130:0/910089301 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f95f402dc30 con 0x7f95dc0a2150 2024-03-23T13:15:07.697 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 7 on host 'smithi130' 2024-03-23T13:15:07.698 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 -- 172.21.15.130:0/910089301 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f95dc0a2150 msgr2=0x7f95dc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f95dc0a2150 0x7f95dc0a4610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f95e400b5c0 tx=0x7f95e40038a0 comp rx=0 tx=0).stop 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 -- 172.21.15.130:0/910089301 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95f4103420 msgr2=0x7f95f419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95f4103420 0x7f95f419edd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f95e800b530 tx=0x7f95e800b840 comp rx=0 tx=0).stop 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 -- 172.21.15.130:0/910089301 shutdown_connections 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f95dc0a2150 0x7f95dc0a4610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95f4105df0 0x7f95f41a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4100ab0 0x7f95f419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 --2- 172.21.15.130:0/910089301 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95f4103420 0x7f95f419edd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 -- 172.21.15.130:0/910089301 >> 172.21.15.130:0/910089301 conn(0x7f95f40fc6d0 msgr2=0x7f95f41041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 -- 172.21.15.130:0/910089301 shutdown_connections 2024-03-23T13:15:07.699 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:15:07.702+0000 7f95f9319700 1 -- 172.21.15.130:0/910089301 wait complete. 2024-03-23T13:15:07.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:07.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:07.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:07.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:07.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:07.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:07.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: from='osd.7 [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T13:15:07.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T13:15:07.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:07.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:07.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:07 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:08.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:08.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:08.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: from='osd.7 [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T13:15:08.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T13:15:08.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:08.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:07 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:08.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:08.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:08.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.285 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: from='osd.7 [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T13:15:08.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T13:15:08.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:08.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:07 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:08.416 DEBUG:teuthology.orchestra.run.smithi130:osd.7> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.7.service 2024-03-23T13:15:08.419 INFO:tasks.cephadm:Deploying osd.8 on smithi155 with /dev/vg_nvme/lv_2... 2024-03-23T13:15:08.420 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T13:15:08.667 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:15:09.121 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:09 smithi155 ceph-mon[36157]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:09.121 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:09 smithi155 ceph-mon[36157]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T13:15:09.121 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:09 smithi155 ceph-mon[36157]: osdmap e49: 8 total, 7 up, 8 in 2024-03-23T13:15:09.121 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:09 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:09.121 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:09 smithi155 ceph-mon[36157]: from='osd.7 [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:15:09.121 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:09 smithi155 ceph-mon[36157]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:15:09.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:09 smithi007 ceph-mon[32184]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:09.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:09 smithi007 ceph-mon[32184]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T13:15:09.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:09 smithi007 ceph-mon[32184]: osdmap e49: 8 total, 7 up, 8 in 2024-03-23T13:15:09.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:09 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:09.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:09 smithi007 ceph-mon[32184]: from='osd.7 [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:15:09.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:09 smithi007 ceph-mon[32184]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:15:09.329 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:09 smithi130 ceph-mon[36225]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:09.329 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:09 smithi130 ceph-mon[36225]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T13:15:09.329 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:09 smithi130 ceph-mon[36225]: osdmap e49: 8 total, 7 up, 8 in 2024-03-23T13:15:09.329 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:09 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:09.329 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:09 smithi130 ceph-mon[36225]: from='osd.7 [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:15:09.329 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:09 smithi130 ceph-mon[36225]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:15:09.629 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:15:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:15:09.473+0000 7efd31321700 -1 osd.7 0 waiting for initial osdmap 2024-03-23T13:15:09.630 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:15:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:15:09.482+0000 7efd2910c700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:15:10.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:10 smithi130 ceph-mon[36225]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:15:10.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:10 smithi130 ceph-mon[36225]: osdmap e50: 8 total, 7 up, 8 in 2024-03-23T13:15:10.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:10.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:10.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:10 smithi130 ceph-mon[36225]: from='osd.7 ' entity='osd.7' 2024-03-23T13:15:10.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:10 smithi130 ceph-mon[36225]: osd.7 [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] boot 2024-03-23T13:15:10.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:10 smithi130 ceph-mon[36225]: osdmap e51: 8 total, 8 up, 8 in 2024-03-23T13:15:10.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:10.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:10 smithi155 ceph-mon[36157]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:15:10.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:10 smithi155 ceph-mon[36157]: osdmap e50: 8 total, 7 up, 8 in 2024-03-23T13:15:10.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:10.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:10.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:10 smithi155 ceph-mon[36157]: from='osd.7 ' entity='osd.7' 2024-03-23T13:15:10.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:10 smithi155 ceph-mon[36157]: osd.7 [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] boot 2024-03-23T13:15:10.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:10 smithi155 ceph-mon[36157]: osdmap e51: 8 total, 8 up, 8 in 2024-03-23T13:15:10.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:10.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:10 smithi007 ceph-mon[32184]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:15:10.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:10 smithi007 ceph-mon[32184]: osdmap e50: 8 total, 7 up, 8 in 2024-03-23T13:15:10.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:10.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:10.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:10 smithi007 ceph-mon[32184]: from='osd.7 ' entity='osd.7' 2024-03-23T13:15:10.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:10 smithi007 ceph-mon[32184]: osd.7 [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] boot 2024-03-23T13:15:10.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:10 smithi007 ceph-mon[32184]: osdmap e51: 8 total, 8 up, 8 in 2024-03-23T13:15:10.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T13:15:10.858 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T13:15:10.859 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T13:15:10.859 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-23T13:15:10.859 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-23T13:15:10.859 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122422 s, 857 MB/s 2024-03-23T13:15:10.859 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-23T13:15:11.542 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-03-23T13:15:11.691 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:11 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:15:11.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:11 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:15:11.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:11 smithi130 ceph-mon[36225]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:11.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:11 smithi130 ceph-mon[36225]: osdmap e52: 8 total, 8 up, 8 in 2024-03-23T13:15:11.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:11 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:11.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:11 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:11.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:11 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:11.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:11 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:11.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:11 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:11.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:11 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:11.784 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:15:11.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:11 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:15:11.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:11 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:15:11.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:11 smithi155 ceph-mon[36157]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:11.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:11 smithi155 ceph-mon[36157]: osdmap e52: 8 total, 8 up, 8 in 2024-03-23T13:15:11.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:11 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:11.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:11 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:11.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:11 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:11.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:11 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:11.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:11 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:11.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:11 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:11.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:11 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:15:11.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:11 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:15:11.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:11 smithi007 ceph-mon[32184]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T13:15:11.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:11 smithi007 ceph-mon[32184]: osdmap e52: 8 total, 8 up, 8 in 2024-03-23T13:15:11.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:11 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:11.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:11 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:11.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:11 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:11.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:11 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:11.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:11 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:11.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:11 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:12.766 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:12 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi130 2024-03-23T13:15:12.767 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:12 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi130 to 5693M 2024-03-23T13:15:12.767 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:12.767 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:12.767 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:12.767 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:12 smithi155 ceph-mon[36157]: osdmap e53: 8 total, 8 up, 8 in 2024-03-23T13:15:12.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:12 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi130 2024-03-23T13:15:12.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:12 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi130 to 5693M 2024-03-23T13:15:12.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:12.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:12.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:12.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:12 smithi007 ceph-mon[32184]: osdmap e53: 8 total, 8 up, 8 in 2024-03-23T13:15:12.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:12 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi130 2024-03-23T13:15:12.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:12 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi130 to 5693M 2024-03-23T13:15:12.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:12.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:12.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:12.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:12 smithi130 ceph-mon[36225]: osdmap e53: 8 total, 8 up, 8 in 2024-03-23T13:15:13.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 -- 172.21.15.155:0/2610248954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac8102cf0 msgr2=0x7f4ac81050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:13.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 --2- 172.21.15.155:0/2610248954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac8102cf0 0x7f4ac81050e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009990 tx=0x7f4ac4009ca0 comp rx=0 tx=0).stop 2024-03-23T13:15:13.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 -- 172.21.15.155:0/2610248954 shutdown_connections 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 --2- 172.21.15.155:0/2610248954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8107f50 0x7f4ac810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 --2- 172.21.15.155:0/2610248954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac8102cf0 0x7f4ac81050e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 --2- 172.21.15.155:0/2610248954 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ac8105620 0x7f4ac8107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 -- 172.21.15.155:0/2610248954 >> 172.21.15.155:0/2610248954 conn(0x7f4ac80fc720 msgr2=0x7f4ac80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 -- 172.21.15.155:0/2610248954 shutdown_connections 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 -- 172.21.15.155:0/2610248954 wait complete. 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 Processor -- start 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.398+0000 7f4acf73a700 1 -- start start 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acf73a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac8102cf0 0x7f4ac8198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acf73a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ac8105620 0x7f4ac81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acf73a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8107f50 0x7f4ac819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acf73a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac819fd00 con 0x7f4ac8107f50 2024-03-23T13:15:13.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acf73a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac819fe40 con 0x7f4ac8105620 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acf73a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac819ffd0 con 0x7f4ac8102cf0 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acd4d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac8102cf0 0x7f4ac8198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acd4d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac8102cf0 0x7f4ac8198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:48808/0 (socket says 172.21.15.155:48808) 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acd4d6700 1 -- 172.21.15.155:0/3397926155 learned_addr learned my addr 172.21.15.155:0/3397926155 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acccd5700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ac8105620 0x7f4ac81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acdcd7700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8107f50 0x7f4ac819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acd4d6700 1 -- 172.21.15.155:0/3397926155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ac8105620 msgr2=0x7f4ac81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acd4d6700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ac8105620 0x7f4ac81986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acd4d6700 1 -- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8107f50 msgr2=0x7f4ac819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acd4d6700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8107f50 0x7f4ac819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acd4d6700 1 -- 172.21.15.155:0/3397926155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac40095f0 con 0x7f4ac8102cf0 2024-03-23T13:15:13.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acccd5700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ac8105620 0x7f4ac81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:15:13.400 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acdcd7700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8107f50 0x7f4ac819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:15:13.400 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acd4d6700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac8102cf0 0x7f4ac8198170 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4000c00 tx=0x7f4ac400f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:15:13.400 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4abe7fc700 1 -- 172.21.15.155:0/3397926155 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac401d070 con 0x7f4ac8102cf0 2024-03-23T13:15:13.400 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4acf73a700 1 -- 172.21.15.155:0/3397926155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac81a0250 con 0x7f4ac8102cf0 2024-03-23T13:15:13.400 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4abe7fc700 1 -- 172.21.15.155:0/3397926155 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac400be60 con 0x7f4ac8102cf0 2024-03-23T13:15:13.400 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.399+0000 7f4abe7fc700 1 -- 172.21.15.155:0/3397926155 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4017630 con 0x7f4ac8102cf0 2024-03-23T13:15:13.400 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.400+0000 7f4acf73a700 1 -- 172.21.15.155:0/3397926155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac81a06c0 con 0x7f4ac8102cf0 2024-03-23T13:15:13.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.401+0000 7f4acf73a700 1 -- 172.21.15.155:0/3397926155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac80fe2a0 con 0x7f4ac8102cf0 2024-03-23T13:15:13.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.402+0000 7f4abe7fc700 1 -- 172.21.15.155:0/3397926155 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f4ac4021980 con 0x7f4ac8102cf0 2024-03-23T13:15:13.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.402+0000 7f4abe7fc700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f4ab40a21d0 0x7f4ab40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:15:13.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.402+0000 7f4abe7fc700 1 -- 172.21.15.155:0/3397926155 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f4ac40ce5f0 con 0x7f4ac8102cf0 2024-03-23T13:15:13.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.403+0000 7f4acccd5700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f4ab40a21d0 0x7f4ab40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:15:13.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.403+0000 7f4acccd5700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f4ab40a21d0 0x7f4ab40a4690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8005950 tx=0x7f4ab80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:15:13.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.406+0000 7f4abe7fc700 1 -- 172.21.15.155:0/3397926155 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4ac40981b0 con 0x7f4ac8102cf0 2024-03-23T13:15:13.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:13.608+0000 7f4acf73a700 1 -- 172.21.15.155:0/3397926155 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4ac802dc60 con 0x7f4ab40a21d0 2024-03-23T13:15:13.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:13 smithi155 ceph-mon[36157]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:13.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:13 smithi155 ceph-mon[36157]: osdmap e54: 8 total, 8 up, 8 in 2024-03-23T13:15:13.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:13 smithi007 ceph-mon[32184]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:13.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:13 smithi007 ceph-mon[32184]: osdmap e54: 8 total, 8 up, 8 in 2024-03-23T13:15:13.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:13 smithi130 ceph-mon[36225]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:13.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:13 smithi130 ceph-mon[36225]: osdmap e54: 8 total, 8 up, 8 in 2024-03-23T13:15:14.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:14 smithi155 ceph-mon[36157]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:15:14.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:15:14.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:15:14.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:14.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:14 smithi155 ceph-mon[36157]: osdmap e55: 8 total, 8 up, 8 in 2024-03-23T13:15:14.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:14 smithi007 ceph-mon[32184]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:15:14.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:15:14.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:15:14.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:14.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:14 smithi007 ceph-mon[32184]: osdmap e55: 8 total, 8 up, 8 in 2024-03-23T13:15:14.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:14 smithi130 ceph-mon[36225]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:15:14.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:15:14.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:15:14.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:14.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:14 smithi130 ceph-mon[36225]: osdmap e55: 8 total, 8 up, 8 in 2024-03-23T13:15:15.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:15 smithi155 ceph-mon[36157]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:15.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:15 smithi007 ceph-mon[32184]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:15.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:15 smithi130 ceph-mon[36225]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:16.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:16 smithi155 ceph-mon[36157]: from='client.? 172.21.15.155:0/3707572257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46014832-6a35-46b2-a918-ef9179dc3bf1"}]: dispatch 2024-03-23T13:15:16.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:16 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46014832-6a35-46b2-a918-ef9179dc3bf1"}]: dispatch 2024-03-23T13:15:16.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:16 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46014832-6a35-46b2-a918-ef9179dc3bf1"}]': finished 2024-03-23T13:15:16.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:16 smithi155 ceph-mon[36157]: osdmap e56: 9 total, 8 up, 9 in 2024-03-23T13:15:16.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:16 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:16.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:16 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/3707572257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46014832-6a35-46b2-a918-ef9179dc3bf1"}]: dispatch 2024-03-23T13:15:16.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:16 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46014832-6a35-46b2-a918-ef9179dc3bf1"}]: dispatch 2024-03-23T13:15:16.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:16 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46014832-6a35-46b2-a918-ef9179dc3bf1"}]': finished 2024-03-23T13:15:16.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:16 smithi007 ceph-mon[32184]: osdmap e56: 9 total, 8 up, 9 in 2024-03-23T13:15:16.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:16 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:16.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:16 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/3707572257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46014832-6a35-46b2-a918-ef9179dc3bf1"}]: dispatch 2024-03-23T13:15:16.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:16 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46014832-6a35-46b2-a918-ef9179dc3bf1"}]: dispatch 2024-03-23T13:15:16.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:16 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46014832-6a35-46b2-a918-ef9179dc3bf1"}]': finished 2024-03-23T13:15:16.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:16 smithi130 ceph-mon[36225]: osdmap e56: 9 total, 8 up, 9 in 2024-03-23T13:15:16.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:16 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:17.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:17 smithi155 ceph-mon[36157]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:17.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:17 smithi155 ceph-mon[36157]: from='client.? 172.21.15.155:0/1698284421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:15:17.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:17 smithi007 ceph-mon[32184]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:17.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:17 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/1698284421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:15:17.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:17 smithi130 ceph-mon[36225]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:17.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:17 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/1698284421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:15:19.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:19 smithi155 ceph-mon[36157]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:19.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:19 smithi007 ceph-mon[32184]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:19.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:19 smithi130 ceph-mon[36225]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:21.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:21 smithi155 ceph-mon[36157]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:21.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:21 smithi007 ceph-mon[32184]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:21.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:21 smithi130 ceph-mon[36225]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:23.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:23 smithi155 ceph-mon[36157]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:23.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:23 smithi007 ceph-mon[32184]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:23.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:23 smithi130 ceph-mon[36225]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:24.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T13:15:24.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:24.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:24 smithi155 ceph-mon[36157]: Deploying daemon osd.8 on smithi155 2024-03-23T13:15:24.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T13:15:24.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:24.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:24 smithi007 ceph-mon[32184]: Deploying daemon osd.8 on smithi155 2024-03-23T13:15:24.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T13:15:24.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:24.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:24 smithi130 ceph-mon[36225]: Deploying daemon osd.8 on smithi155 2024-03-23T13:15:25.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:25 smithi155 ceph-mon[36157]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:25.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:25 smithi007 ceph-mon[32184]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:25.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:25 smithi130 ceph-mon[36225]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:27.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:27 smithi155 ceph-mon[36157]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:27.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:27 smithi007 ceph-mon[32184]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:27.941 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:27 smithi130 ceph-mon[36225]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:29.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:29 smithi155 ceph-mon[36157]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:29.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:29 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:29.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:29 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:29.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:29 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:29.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:29 smithi007 ceph-mon[32184]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:29.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:29 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:29.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:29 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:29.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:29 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:29.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:29 smithi130 ceph-mon[36225]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:29.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:29 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:29.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:29 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:29.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:29 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:31.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:31 smithi155 ceph-mon[36157]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:31.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:31 smithi007 ceph-mon[32184]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:31.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:31 smithi130 ceph-mon[36225]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:32.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.978+0000 7f4abe7fc700 1 -- 172.21.15.155:0/3397926155 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4ac802dc60 con 0x7f4ab40a21d0 2024-03-23T13:15:32.976 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 8 on host 'smithi155' 2024-03-23T13:15:32.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.980+0000 7f4acf73a700 1 -- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f4ab40a21d0 msgr2=0x7f4ab40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:32.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.980+0000 7f4acf73a700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f4ab40a21d0 0x7f4ab40a4690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8005950 tx=0x7f4ab80058e0 comp rx=0 tx=0).stop 2024-03-23T13:15:32.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.980+0000 7f4acf73a700 1 -- 172.21.15.155:0/3397926155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac8102cf0 msgr2=0x7f4ac8198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:32.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.980+0000 7f4acf73a700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac8102cf0 0x7f4ac8198170 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4000c00 tx=0x7f4ac400f740 comp rx=0 tx=0).stop 2024-03-23T13:15:32.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.980+0000 7f4acf73a700 1 -- 172.21.15.155:0/3397926155 shutdown_connections 2024-03-23T13:15:32.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.980+0000 7f4acf73a700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f4ab40a21d0 0x7f4ab40a4690 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:32.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.980+0000 7f4acf73a700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8107f50 0x7f4ac819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:32.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.981+0000 7f4acf73a700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac8102cf0 0x7f4ac8198170 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:32.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.981+0000 7f4acf73a700 1 --2- 172.21.15.155:0/3397926155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ac8105620 0x7f4ac81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:32.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.981+0000 7f4acf73a700 1 -- 172.21.15.155:0/3397926155 >> 172.21.15.155:0/3397926155 conn(0x7f4ac80fc720 msgr2=0x7f4ac81035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:15:32.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.981+0000 7f4acf73a700 1 -- 172.21.15.155:0/3397926155 shutdown_connections 2024-03-23T13:15:32.979 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:15:32.981+0000 7f4acf73a700 1 -- 172.21.15.155:0/3397926155 wait complete. 2024-03-23T13:15:33.622 DEBUG:teuthology.orchestra.run.smithi155:osd.8> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.8.service 2024-03-23T13:15:33.624 INFO:tasks.cephadm:Deploying osd.9 on smithi007 with /dev/vg_nvme/lv_1... 2024-03-23T13:15:33.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T13:15:33.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:33 smithi155 ceph-mon[36157]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:33.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:33 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:33 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:33 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:33 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:33 smithi155 ceph-mon[36157]: from='osd.8 [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T13:15:33.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:33 smithi155 ceph-mon[36157]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T13:15:33.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:33 smithi007 ceph-mon[32184]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:33.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:33 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:33 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:33 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:33 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:33 smithi007 ceph-mon[32184]: from='osd.8 [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T13:15:33.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:33 smithi007 ceph-mon[32184]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T13:15:33.875 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:15:33.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:33 smithi130 ceph-mon[36225]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:33.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:33 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:33 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:33 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:33 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:33.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:33 smithi130 ceph-mon[36225]: from='osd.8 [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T13:15:33.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:33 smithi130 ceph-mon[36225]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T13:15:35.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:35 smithi007 ceph-mon[32184]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:35.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:35 smithi007 ceph-mon[32184]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T13:15:35.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:35 smithi007 ceph-mon[32184]: osdmap e57: 9 total, 8 up, 9 in 2024-03-23T13:15:35.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:35.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:35 smithi007 ceph-mon[32184]: from='osd.8 [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:15:35.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:35 smithi007 ceph-mon[32184]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:15:35.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:35 smithi155 ceph-mon[36157]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:35.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:35 smithi155 ceph-mon[36157]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T13:15:35.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:35 smithi155 ceph-mon[36157]: osdmap e57: 9 total, 8 up, 9 in 2024-03-23T13:15:35.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:35.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:35 smithi155 ceph-mon[36157]: from='osd.8 [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:15:35.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:35 smithi155 ceph-mon[36157]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:15:35.536 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:15:35 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:15:35.265+0000 7f2f67fc3700 -1 osd.8 0 waiting for initial osdmap 2024-03-23T13:15:35.537 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:15:35 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:15:35.274+0000 7f2f63db6700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:15:35.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:35 smithi130 ceph-mon[36225]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:35.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:35 smithi130 ceph-mon[36225]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T13:15:35.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:35 smithi130 ceph-mon[36225]: osdmap e57: 9 total, 8 up, 9 in 2024-03-23T13:15:35.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:35.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:35 smithi130 ceph-mon[36225]: from='osd.8 [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:15:35.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:35 smithi130 ceph-mon[36225]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:15:35.937 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T13:15:35.938 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T13:15:35.938 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-23T13:15:35.938 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-23T13:15:35.938 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00834055 s, 1.3 GB/s 2024-03-23T13:15:35.938 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-23T13:15:36.535 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:36 smithi155 ceph-mon[36157]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:15:36.535 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:36 smithi155 ceph-mon[36157]: osdmap e58: 9 total, 8 up, 9 in 2024-03-23T13:15:36.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:36 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:36.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:36 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:36.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:36 smithi007 ceph-mon[32184]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:15:36.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:36 smithi007 ceph-mon[32184]: osdmap e58: 9 total, 8 up, 9 in 2024-03-23T13:15:36.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:36 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:36.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:36 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:36.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_1 2024-03-23T13:15:36.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:36 smithi130 ceph-mon[36225]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:15:36.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:36 smithi130 ceph-mon[36225]: osdmap e58: 9 total, 8 up, 9 in 2024-03-23T13:15:36.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:36.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:36 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:15:37.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:15:37.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:15:37.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:37.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:37.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: osd.8 [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] boot 2024-03-23T13:15:37.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T13:15:37.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:37.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi155 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi155 to 5693M 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:15:37.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:15:37.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:37.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:37.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: osd.8 [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] boot 2024-03-23T13:15:37.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T13:15:37.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:37.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi155 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi155 to 5693M 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:37.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:37.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:37.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:15:37.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: osd.8 [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] boot 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi155 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:15:37.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi155 to 5693M 2024-03-23T13:15:37.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:37.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:37.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:37.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:37.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:37.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:37.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:38.535 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:38 smithi155 ceph-mon[36157]: osdmap e60: 9 total, 9 up, 9 in 2024-03-23T13:15:38.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:38 smithi007 ceph-mon[32184]: osdmap e60: 9 total, 9 up, 9 in 2024-03-23T13:15:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.596+0000 7fd5eac48700 1 -- 172.21.15.7:0/3798133906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e4102330 msgr2=0x7fd5e4102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.596+0000 7fd5eac48700 1 --2- 172.21.15.7:0/3798133906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e4102330 0x7fd5e4102790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00099c0 tx=0x7fd5d0009cd0 comp rx=0 tx=0).stop 2024-03-23T13:15:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 -- 172.21.15.7:0/3798133906 shutdown_connections 2024-03-23T13:15:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 --2- 172.21.15.7:0/3798133906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e4102cd0 0x7fd5e4109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 --2- 172.21.15.7:0/3798133906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5e4101130 0x7fd5e4101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 --2- 172.21.15.7:0/3798133906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e4102330 0x7fd5e4102790 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 -- 172.21.15.7:0/3798133906 >> 172.21.15.7:0/3798133906 conn(0x7fd5e40fc6f0 msgr2=0x7fd5e40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 -- 172.21.15.7:0/3798133906 shutdown_connections 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 -- 172.21.15.7:0/3798133906 wait complete. 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 Processor -- start 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 -- start start 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e4101130 0x7fd5e4197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5e4102330 0x7fd5e41976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e4102cd0 0x7fd5e419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e419ed20 con 0x7fd5e4102cd0 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e419ee60 con 0x7fd5e4102330 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.597+0000 7fd5eac48700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e419eff0 con 0x7fd5e4101130 2024-03-23T13:15:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e91e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e4102cd0 0x7fd5e419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:15:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e91e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e4102cd0 0x7fd5e419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60092/0 (socket says 172.21.15.7:60092) 2024-03-23T13:15:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e91e5700 1 -- 172.21.15.7:0/1925940253 learned_addr learned my addr 172.21.15.7:0/1925940253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:15:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e3fff700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5e4102330 0x7fd5e41976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:15:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e89e4700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e4101130 0x7fd5e4197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:15:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e91e5700 1 -- 172.21.15.7:0/1925940253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e4101130 msgr2=0x7fd5e4197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e91e5700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e4101130 0x7fd5e4197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e91e5700 1 -- 172.21.15.7:0/1925940253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5e4102330 msgr2=0x7fd5e41976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e91e5700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5e4102330 0x7fd5e41976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e91e5700 1 -- 172.21.15.7:0/1925940253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d00095f0 con 0x7fd5e4102cd0 2024-03-23T13:15:38.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e3fff700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5e4102330 0x7fd5e41976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:15:38.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e91e5700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e4102cd0 0x7fd5e419e750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00b9a0 tx=0x7fd5dc00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:15:38.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e1ffb700 1 -- 172.21.15.7:0/1925940253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00c9b0 con 0x7fd5e4102cd0 2024-03-23T13:15:38.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e1ffb700 1 -- 172.21.15.7:0/1925940253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5dc014440 con 0x7fd5e4102cd0 2024-03-23T13:15:38.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5eac48700 1 -- 172.21.15.7:0/1925940253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e406aa40 con 0x7fd5e4102cd0 2024-03-23T13:15:38.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.598+0000 7fd5e1ffb700 1 -- 172.21.15.7:0/1925940253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0075d0 con 0x7fd5e4102cd0 2024-03-23T13:15:38.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.599+0000 7fd5eac48700 1 -- 172.21.15.7:0/1925940253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e406af70 con 0x7fd5e4102cd0 2024-03-23T13:15:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.599+0000 7fd5eac48700 1 -- 172.21.15.7:0/1925940253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e4068ec0 con 0x7fd5e4102cd0 2024-03-23T13:15:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.600+0000 7fd5e1ffb700 1 -- 172.21.15.7:0/1925940253 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fd5dc014920 con 0x7fd5e4102cd0 2024-03-23T13:15:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.601+0000 7fd5e1ffb700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5d40a2150 0x7fd5d40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:15:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.601+0000 7fd5e89e4700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5d40a2150 0x7fd5d40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:15:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.601+0000 7fd5e1ffb700 1 -- 172.21.15.7:0/1925940253 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7fd5dc006e40 con 0x7fd5e4102cd0 2024-03-23T13:15:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.601+0000 7fd5e89e4700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5d40a2150 0x7fd5d40a4610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8006fd0 tx=0x7fd5d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:15:38.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.606+0000 7fd5e1ffb700 1 -- 172.21.15.7:0/1925940253 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd5dc096440 con 0x7fd5e4102cd0 2024-03-23T13:15:38.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:38 smithi130 ceph-mon[36225]: osdmap e60: 9 total, 9 up, 9 in 2024-03-23T13:15:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:38.824+0000 7fd5eac48700 1 -- 172.21.15.7:0/1925940253 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd5e4108060 con 0x7fd5d40a2150 2024-03-23T13:15:39.535 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:39 smithi155 ceph-mon[36157]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:39.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:39 smithi155 ceph-mon[36157]: osdmap e61: 9 total, 9 up, 9 in 2024-03-23T13:15:39.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:39 smithi155 ceph-mon[36157]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:15:39.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:15:39.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:15:39.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:39.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:39 smithi007 ceph-mon[32184]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:39.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:39 smithi007 ceph-mon[32184]: osdmap e61: 9 total, 9 up, 9 in 2024-03-23T13:15:39.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:39 smithi007 ceph-mon[32184]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:15:39.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:15:39.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:15:39.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:39.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:39 smithi130 ceph-mon[36225]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:39.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:39 smithi130 ceph-mon[36225]: osdmap e61: 9 total, 9 up, 9 in 2024-03-23T13:15:39.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:39 smithi130 ceph-mon[36225]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:15:39.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:15:39.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:15:39.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:40.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:40 smithi007 ceph-mon[32184]: osdmap e62: 9 total, 9 up, 9 in 2024-03-23T13:15:40.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:40 smithi130 ceph-mon[36225]: osdmap e62: 9 total, 9 up, 9 in 2024-03-23T13:15:40.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:40 smithi155 ceph-mon[36157]: osdmap e62: 9 total, 9 up, 9 in 2024-03-23T13:15:41.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:41 smithi007 ceph-mon[32184]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:41.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:41 smithi007 ceph-mon[32184]: osdmap e63: 9 total, 9 up, 9 in 2024-03-23T13:15:41.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:41 smithi130 ceph-mon[36225]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:41.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:41 smithi130 ceph-mon[36225]: osdmap e63: 9 total, 9 up, 9 in 2024-03-23T13:15:41.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:41 smithi155 ceph-mon[36157]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:41.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:41 smithi155 ceph-mon[36157]: osdmap e63: 9 total, 9 up, 9 in 2024-03-23T13:15:42.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:42 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1562597877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fdeaf21-f0a5-4dff-ac58-8bb059972ffe"}]: dispatch 2024-03-23T13:15:42.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:42 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1562597877' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fdeaf21-f0a5-4dff-ac58-8bb059972ffe"}]': finished 2024-03-23T13:15:42.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:42 smithi007 ceph-mon[32184]: osdmap e64: 10 total, 9 up, 10 in 2024-03-23T13:15:42.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:15:42.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:42 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/722082965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:15:42.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:42 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1562597877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fdeaf21-f0a5-4dff-ac58-8bb059972ffe"}]: dispatch 2024-03-23T13:15:42.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:42 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1562597877' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fdeaf21-f0a5-4dff-ac58-8bb059972ffe"}]': finished 2024-03-23T13:15:42.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:42 smithi130 ceph-mon[36225]: osdmap e64: 10 total, 9 up, 10 in 2024-03-23T13:15:42.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:15:42.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:42 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/722082965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:15:42.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:42 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1562597877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fdeaf21-f0a5-4dff-ac58-8bb059972ffe"}]: dispatch 2024-03-23T13:15:42.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:42 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1562597877' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fdeaf21-f0a5-4dff-ac58-8bb059972ffe"}]': finished 2024-03-23T13:15:42.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:42 smithi155 ceph-mon[36157]: osdmap e64: 10 total, 9 up, 10 in 2024-03-23T13:15:42.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:15:42.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:42 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/722082965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:15:43.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:43 smithi130 ceph-mon[36225]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:15:44.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:43 smithi155 ceph-mon[36157]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:15:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:43 smithi007 ceph-mon[32184]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:15:45.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:45 smithi130 ceph-mon[36225]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-23T13:15:46.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:45 smithi007 ceph-mon[32184]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-23T13:15:46.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:45 smithi155 ceph-mon[36157]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-23T13:15:47.899 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:47 smithi007 ceph-mon[32184]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-23T13:15:47.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:47 smithi130 ceph-mon[36225]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-23T13:15:48.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:47 smithi155 ceph-mon[36157]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-23T13:15:49.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:49 smithi130 ceph-mon[36225]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T13:15:49.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:49 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T13:15:49.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:49 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:49.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:49 smithi007 ceph-mon[32184]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T13:15:49.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:49 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T13:15:49.957 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:49 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:50.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:49 smithi155 ceph-mon[36157]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T13:15:50.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:49 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T13:15:50.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:49 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:50.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:50 smithi130 ceph-mon[36225]: Deploying daemon osd.9 on smithi007 2024-03-23T13:15:51.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:50 smithi007 ceph-mon[32184]: Deploying daemon osd.9 on smithi007 2024-03-23T13:15:51.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:50 smithi155 ceph-mon[36157]: Deploying daemon osd.9 on smithi007 2024-03-23T13:15:51.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:51 smithi007 ceph-mon[32184]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:51.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:51 smithi130 ceph-mon[36225]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:52.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:51 smithi155 ceph-mon[36157]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:53.751 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:53 smithi007 ceph-mon[32184]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:53.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:53 smithi130 ceph-mon[36225]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:54.035 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:53 smithi155 ceph-mon[36157]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:55.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:55 smithi130 ceph-mon[36225]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:55.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:55.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:55.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:55.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:55 smithi007 ceph-mon[32184]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:55.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:55.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:55.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:56.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:55 smithi155 ceph-mon[36157]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:56.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:56.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:56.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:57.906 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:57 smithi007 ceph-mon[32184]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:57.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:57 smithi130 ceph-mon[36225]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:58.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:57 smithi155 ceph-mon[36157]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:59.130 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 9 on host 'smithi007' 2024-03-23T13:15:59.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.132+0000 7fd5e1ffb700 1 -- 172.21.15.7:0/1925940253 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd5e4108060 con 0x7fd5d40a2150 2024-03-23T13:15:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 -- 172.21.15.7:0/1925940253 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5d40a2150 msgr2=0x7fd5d40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5d40a2150 0x7fd5d40a4610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8006fd0 tx=0x7fd5d8008040 comp rx=0 tx=0).stop 2024-03-23T13:15:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 -- 172.21.15.7:0/1925940253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e4102cd0 msgr2=0x7fd5e419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:15:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e4102cd0 0x7fd5e419e750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00b9a0 tx=0x7fd5dc00bd60 comp rx=0 tx=0).stop 2024-03-23T13:15:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 -- 172.21.15.7:0/1925940253 shutdown_connections 2024-03-23T13:15:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5d40a2150 0x7fd5d40a4610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e4102cd0 0x7fd5e419e750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e4101130 0x7fd5e4197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 --2- 172.21.15.7:0/1925940253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5e4102330 0x7fd5e41976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:15:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 -- 172.21.15.7:0/1925940253 >> 172.21.15.7:0/1925940253 conn(0x7fd5e40fc6f0 msgr2=0x7fd5e4105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:15:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 -- 172.21.15.7:0/1925940253 shutdown_connections 2024-03-23T13:15:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:15:59.134+0000 7fd5eac48700 1 -- 172.21.15.7:0/1925940253 wait complete. 2024-03-23T13:15:59.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:59 smithi007 ceph-mon[32184]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:59.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:59.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:59.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:59 smithi007 ceph-mon[32184]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T13:15:59.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:15:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:59.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:59 smithi155 ceph-mon[36157]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:59.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:59.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:59.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:59 smithi155 ceph-mon[36157]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T13:15:59.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:15:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:59.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:59 smithi130 ceph-mon[36225]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:15:59.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:15:59.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:15:59.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:59 smithi130 ceph-mon[36225]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T13:15:59.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:15:59.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:15:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:15:59.749 DEBUG:teuthology.orchestra.run.smithi007:osd.9> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.9.service 2024-03-23T13:15:59.751 INFO:tasks.cephadm:Deploying osd.10 on smithi130 with /dev/vg_nvme/lv_1... 2024-03-23T13:15:59.751 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T13:15:59.999 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:16:00.692 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:00 smithi130 ceph-mon[36225]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T13:16:00.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:00 smithi130 ceph-mon[36225]: osdmap e65: 10 total, 9 up, 10 in 2024-03-23T13:16:00.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:00 smithi130 ceph-mon[36225]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:16:00.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:00 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:00.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:00 smithi007 ceph-mon[32184]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T13:16:00.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:00 smithi007 ceph-mon[32184]: osdmap e65: 10 total, 9 up, 10 in 2024-03-23T13:16:00.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:00 smithi007 ceph-mon[32184]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:16:00.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:00 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:01.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:00 smithi155 ceph-mon[36157]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T13:16:01.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:00 smithi155 ceph-mon[36157]: osdmap e65: 10 total, 9 up, 10 in 2024-03-23T13:16:01.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:00 smithi155 ceph-mon[36157]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-23T13:16:01.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:00 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:01.239 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:16:01 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:16:01.153+0000 7f05cb3ae700 -1 osd.9 0 waiting for initial osdmap 2024-03-23T13:16:01.239 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:16:01 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:16:01.164+0000 7f05c2998700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:16:01.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:01 smithi130 ceph-mon[36225]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:16:01.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:01 smithi130 ceph-mon[36225]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:16:01.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:01 smithi130 ceph-mon[36225]: osdmap e66: 10 total, 9 up, 10 in 2024-03-23T13:16:01.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:01 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:01.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:01 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:01.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:01 smithi130 ceph-mon[36225]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' 2024-03-23T13:16:01.960 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T13:16:01.961 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T13:16:01.961 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-03-23T13:16:01.961 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-03-23T13:16:01.961 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137711 s, 761 MB/s 2024-03-23T13:16:01.961 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-03-23T13:16:02.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:01 smithi155 ceph-mon[36157]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:16:02.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:01 smithi155 ceph-mon[36157]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:16:02.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:01 smithi155 ceph-mon[36157]: osdmap e66: 10 total, 9 up, 10 in 2024-03-23T13:16:02.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:01 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:02.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:01 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:02.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:01 smithi155 ceph-mon[36157]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' 2024-03-23T13:16:02.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:01 smithi007 ceph-mon[32184]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:16:02.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:01 smithi007 ceph-mon[32184]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-23T13:16:02.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:01 smithi007 ceph-mon[32184]: osdmap e66: 10 total, 9 up, 10 in 2024-03-23T13:16:02.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:01 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:02.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:01 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:02.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:01 smithi007 ceph-mon[32184]: from='osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456]' entity='osd.9' 2024-03-23T13:16:02.622 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi130:vg_nvme/lv_1 2024-03-23T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:02 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:02 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:02 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:02 smithi007 ceph-mon[32184]: osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] boot 2024-03-23T13:16:02.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:02 smithi007 ceph-mon[32184]: osdmap e67: 10 total, 10 up, 10 in 2024-03-23T13:16:02.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:02 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:02.873 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:16:02.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:02 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:16:02.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:02 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:16:02.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:02 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:02.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:02 smithi130 ceph-mon[36225]: osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] boot 2024-03-23T13:16:02.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:02 smithi130 ceph-mon[36225]: osdmap e67: 10 total, 10 up, 10 in 2024-03-23T13:16:02.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:02 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:03.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:02 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:16:03.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:02 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:16:03.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:02 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:03.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:02 smithi155 ceph-mon[36157]: osd.9 [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] boot 2024-03-23T13:16:03.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:02 smithi155 ceph-mon[36157]: osdmap e67: 10 total, 10 up, 10 in 2024-03-23T13:16:03.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:02 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi007 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi007 to 4273M 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:04.011 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:04.012 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:16:04.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi007 2024-03-23T13:16:04.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi007 to 4273M 2024-03-23T13:16:04.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:04.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:04.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T13:16:04.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi007 2024-03-23T13:16:04.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:04.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi007 to 4273M 2024-03-23T13:16:04.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:04.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:04.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:04.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.569+0000 7ff930c12700 1 -- 172.21.15.130:0/2901565218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff92c102260 msgr2=0x7ff92c102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:04.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.569+0000 7ff930c12700 1 --2- 172.21.15.130:0/2901565218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff92c102260 0x7ff92c102680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0099c0 tx=0x7ff91c009cd0 comp rx=0 tx=0).stop 2024-03-23T13:16:04.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.569+0000 7ff930c12700 1 -- 172.21.15.130:0/2901565218 shutdown_connections 2024-03-23T13:16:04.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.569+0000 7ff930c12700 1 --2- 172.21.15.130:0/2901565218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c103e00 0x7ff92c10a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:04.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.569+0000 7ff930c12700 1 --2- 172.21.15.130:0/2901565218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff92c102260 0x7ff92c102680 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:04.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.569+0000 7ff930c12700 1 --2- 172.21.15.130:0/2901565218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c103460 0x7ff92c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:04.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.569+0000 7ff930c12700 1 -- 172.21.15.130:0/2901565218 >> 172.21.15.130:0/2901565218 conn(0x7ff92c0fd800 msgr2=0x7ff92c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:16:04.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.569+0000 7ff930c12700 1 -- 172.21.15.130:0/2901565218 shutdown_connections 2024-03-23T13:16:04.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.569+0000 7ff930c12700 1 -- 172.21.15.130:0/2901565218 wait complete. 2024-03-23T13:16:04.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.569+0000 7ff930c12700 1 Processor -- start 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff930c12700 1 -- start start 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff930c12700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c102260 0x7ff92c070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff930c12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c103460 0x7ff92c070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff930c12700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff92c103e00 0x7ff92c1a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff930c12700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c073f90 con 0x7ff92c103460 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff930c12700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c074100 con 0x7ff92c103e00 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff930c12700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c0742c0 con 0x7ff92c102260 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff92ad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff92c103e00 0x7ff92c1a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff92ad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff92c103e00 0x7ff92c1a8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54442/0 (socket says 172.21.15.130:54442) 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff92ad9d700 1 -- 172.21.15.130:0/322070716 learned_addr learned my addr 172.21.15.130:0/322070716 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff92a59c700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c102260 0x7ff92c070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff921d9b700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c103460 0x7ff92c070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:04.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff92ad9d700 1 -- 172.21.15.130:0/322070716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c102260 msgr2=0x7ff92c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff92ad9d700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c102260 0x7ff92c070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff92ad9d700 1 -- 172.21.15.130:0/322070716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c103460 msgr2=0x7ff92c070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff92ad9d700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c103460 0x7ff92c070ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff92ad9d700 1 -- 172.21.15.130:0/322070716 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c0095f0 con 0x7ff92c103e00 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.570+0000 7ff92ad9d700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff92c103e00 0x7ff92c1a8430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff924007d70 tx=0x7ff92400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.571+0000 7ff923fff700 1 -- 172.21.15.130:0/322070716 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92400ad40 con 0x7ff92c103e00 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.571+0000 7ff923fff700 1 -- 172.21.15.130:0/322070716 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff924012900 con 0x7ff92c103e00 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.571+0000 7ff923fff700 1 -- 172.21.15.130:0/322070716 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924011780 con 0x7ff92c103e00 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.571+0000 7ff930c12700 1 -- 172.21.15.130:0/322070716 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92c074550 con 0x7ff92c103e00 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.571+0000 7ff930c12700 1 -- 172.21.15.130:0/322070716 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92c074aa0 con 0x7ff92c103e00 2024-03-23T13:16:04.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.572+0000 7ff930c12700 1 -- 172.21.15.130:0/322070716 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92c068ec0 con 0x7ff92c103e00 2024-03-23T13:16:04.572 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.573+0000 7ff923fff700 1 -- 172.21.15.130:0/322070716 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7ff924012a70 con 0x7ff92c103e00 2024-03-23T13:16:04.572 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.573+0000 7ff923fff700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9180a21d0 0x7ff9180a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:04.572 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.573+0000 7ff923fff700 1 -- 172.21.15.130:0/322070716 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7ff9240cdaa0 con 0x7ff92c103e00 2024-03-23T13:16:04.572 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.573+0000 7ff92a59c700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9180a21d0 0x7ff9180a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:04.572 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.574+0000 7ff92a59c700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9180a21d0 0x7ff9180a4690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0038c0 tx=0x7ff91c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:16:04.575 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.577+0000 7ff923fff700 1 -- 172.21.15.130:0/322070716 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff924097270 con 0x7ff92c103e00 2024-03-23T13:16:04.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:04.779+0000 7ff930c12700 1 -- 172.21.15.130:0/322070716 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff92c02dbd0 con 0x7ff9180a21d0 2024-03-23T13:16:05.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:04 smithi007 ceph-mon[32184]: osdmap e68: 10 total, 10 up, 10 in 2024-03-23T13:16:05.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:04 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:16:05.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:04 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:16:05.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:04 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:05.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:04 smithi130 ceph-mon[36225]: osdmap e68: 10 total, 10 up, 10 in 2024-03-23T13:16:05.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:04 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:16:05.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:04 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:16:05.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:04 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:05.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:04 smithi155 ceph-mon[36157]: osdmap e68: 10 total, 10 up, 10 in 2024-03-23T13:16:05.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:04 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:16:05.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:04 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:16:05.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:04 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:05.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:05 smithi130 ceph-mon[36225]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:05.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:05 smithi130 ceph-mon[36225]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:16:06.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:05 smithi007 ceph-mon[32184]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:06.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:05 smithi007 ceph-mon[32184]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:16:06.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:05 smithi155 ceph-mon[36157]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:06.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:05 smithi155 ceph-mon[36157]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:16:08.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:07 smithi007 ceph-mon[32184]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:08.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:07 smithi007 ceph-mon[32184]: from='client.? 172.21.15.130:0/38115893' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ebd1ee-dd0b-424f-9e4d-113d31950c5f"}]: dispatch 2024-03-23T13:16:08.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:07 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ebd1ee-dd0b-424f-9e4d-113d31950c5f"}]: dispatch 2024-03-23T13:16:08.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:07 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06ebd1ee-dd0b-424f-9e4d-113d31950c5f"}]': finished 2024-03-23T13:16:08.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:07 smithi007 ceph-mon[32184]: osdmap e69: 11 total, 10 up, 11 in 2024-03-23T13:16:08.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:07 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:08.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:07 smithi130 ceph-mon[36225]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:08.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:07 smithi130 ceph-mon[36225]: from='client.? 172.21.15.130:0/38115893' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ebd1ee-dd0b-424f-9e4d-113d31950c5f"}]: dispatch 2024-03-23T13:16:08.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:07 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ebd1ee-dd0b-424f-9e4d-113d31950c5f"}]: dispatch 2024-03-23T13:16:08.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:07 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06ebd1ee-dd0b-424f-9e4d-113d31950c5f"}]': finished 2024-03-23T13:16:08.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:07 smithi130 ceph-mon[36225]: osdmap e69: 11 total, 10 up, 11 in 2024-03-23T13:16:08.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:07 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:08.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:07 smithi155 ceph-mon[36157]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:08.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:07 smithi155 ceph-mon[36157]: from='client.? 172.21.15.130:0/38115893' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ebd1ee-dd0b-424f-9e4d-113d31950c5f"}]: dispatch 2024-03-23T13:16:08.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:07 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ebd1ee-dd0b-424f-9e4d-113d31950c5f"}]: dispatch 2024-03-23T13:16:08.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:07 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06ebd1ee-dd0b-424f-9e4d-113d31950c5f"}]': finished 2024-03-23T13:16:08.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:07 smithi155 ceph-mon[36157]: osdmap e69: 11 total, 10 up, 11 in 2024-03-23T13:16:08.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:07 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:09.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:08 smithi007 ceph-mon[32184]: from='client.? 172.21.15.130:0/2705993706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:16:09.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:08 smithi130 ceph-mon[36225]: from='client.? 172.21.15.130:0/2705993706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:16:09.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:08 smithi155 ceph-mon[36157]: from='client.? 172.21.15.130:0/2705993706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:16:10.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:09 smithi007 ceph-mon[32184]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:10.192 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:09 smithi130 ceph-mon[36225]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:10.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:09 smithi155 ceph-mon[36157]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:11.942 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:11 smithi130 ceph-mon[36225]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:12.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:11 smithi007 ceph-mon[32184]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:12.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:11 smithi155 ceph-mon[36157]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:13.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:13 smithi130 ceph-mon[36225]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:14.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:13 smithi007 ceph-mon[32184]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:14.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:13 smithi155 ceph-mon[36157]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:15.909 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:15 smithi130 ceph-mon[36225]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:15.909 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:15 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T13:16:15.909 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:15 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:16.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:15 smithi007 ceph-mon[32184]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:16.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:15 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T13:16:16.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:15 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:16.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:15 smithi155 ceph-mon[36157]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:16.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:15 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T13:16:16.089 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:15 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:17.031 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:16 smithi130 ceph-mon[36225]: Deploying daemon osd.10 on smithi130 2024-03-23T13:16:17.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:16 smithi155 ceph-mon[36157]: Deploying daemon osd.10 on smithi130 2024-03-23T13:16:17.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:16 smithi007 ceph-mon[32184]: Deploying daemon osd.10 on smithi130 2024-03-23T13:16:18.148 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:17 smithi130 ceph-mon[36225]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:18.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:17 smithi155 ceph-mon[36157]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:18.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:17 smithi007 ceph-mon[32184]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:19.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:19 smithi130 ceph-mon[36225]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:20.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:19 smithi155 ceph-mon[36157]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:20.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:19 smithi007 ceph-mon[32184]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:21.865 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:21 smithi130 ceph-mon[36225]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:21.865 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:21.865 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:21.865 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:22.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:21 smithi155 ceph-mon[36157]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:22.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:22.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:22.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:22.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:21 smithi007 ceph-mon[32184]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:22.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:22.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:22.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:24.091 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:23 smithi130 ceph-mon[36225]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:24.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:23 smithi155 ceph-mon[36157]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:24.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:23 smithi007 ceph-mon[32184]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:24.798 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.799+0000 7ff923fff700 1 -- 172.21.15.130:0/322070716 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff92c02dbd0 con 0x7ff9180a21d0 2024-03-23T13:16:24.798 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 10 on host 'smithi130' 2024-03-23T13:16:24.800 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.802+0000 7ff930c12700 1 -- 172.21.15.130:0/322070716 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9180a21d0 msgr2=0x7ff9180a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:24.800 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.802+0000 7ff930c12700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9180a21d0 0x7ff9180a4690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0038c0 tx=0x7ff91c00b540 comp rx=0 tx=0).stop 2024-03-23T13:16:24.800 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.802+0000 7ff930c12700 1 -- 172.21.15.130:0/322070716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff92c103e00 msgr2=0x7ff92c1a8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:24.800 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.802+0000 7ff930c12700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff92c103e00 0x7ff92c1a8430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff924007d70 tx=0x7ff92400a340 comp rx=0 tx=0).stop 2024-03-23T13:16:24.801 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.802+0000 7ff930c12700 1 -- 172.21.15.130:0/322070716 shutdown_connections 2024-03-23T13:16:24.801 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.802+0000 7ff930c12700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9180a21d0 0x7ff9180a4690 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:24.801 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.802+0000 7ff930c12700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff92c103e00 0x7ff92c1a8430 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:24.801 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.802+0000 7ff930c12700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c102260 0x7ff92c070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:24.801 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.802+0000 7ff930c12700 1 --2- 172.21.15.130:0/322070716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c103460 0x7ff92c070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:24.801 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.802+0000 7ff930c12700 1 -- 172.21.15.130:0/322070716 >> 172.21.15.130:0/322070716 conn(0x7ff92c0fd800 msgr2=0x7ff92c106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:16:24.801 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.803+0000 7ff930c12700 1 -- 172.21.15.130:0/322070716 shutdown_connections 2024-03-23T13:16:24.801 INFO:teuthology.orchestra.run.smithi130.stderr:2024-03-23T13:16:24.803+0000 7ff930c12700 1 -- 172.21.15.130:0/322070716 wait complete. 2024-03-23T13:16:25.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:25.286 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:25.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:25.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:25.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: from='osd.10 [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T13:16:25.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:24 smithi155 ceph-mon[36157]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T13:16:25.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:25.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:25.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:25.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.325 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:25.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: from='osd.10 [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T13:16:25.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:24 smithi007 ceph-mon[32184]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: from='osd.10 [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T13:16:25.420 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:24 smithi130 ceph-mon[36225]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T13:16:25.452 DEBUG:teuthology.orchestra.run.smithi130:osd.10> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.10.service 2024-03-23T13:16:25.454 INFO:tasks.cephadm:Deploying osd.11 on smithi155 with /dev/vg_nvme/lv_1... 2024-03-23T13:16:25.454 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T13:16:25.701 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:16:26.906 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:26 smithi130 ceph-mon[36225]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T13:16:26.907 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:26 smithi130 ceph-mon[36225]: osdmap e70: 11 total, 10 up, 11 in 2024-03-23T13:16:26.907 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:26 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:26.907 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:26 smithi130 ceph-mon[36225]: from='osd.10 [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:16:26.907 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:26 smithi130 ceph-mon[36225]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:16:26.907 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:16:26 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:16:26.778+0000 7fa8984ab700 -1 osd.10 0 waiting for initial osdmap 2024-03-23T13:16:26.907 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:16:26 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:16:26.789+0000 7fa89429e700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:16:27.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:26 smithi155 ceph-mon[36157]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T13:16:27.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:26 smithi155 ceph-mon[36157]: osdmap e70: 11 total, 10 up, 11 in 2024-03-23T13:16:27.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:26 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:27.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:26 smithi155 ceph-mon[36157]: from='osd.10 [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:16:27.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:26 smithi155 ceph-mon[36157]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:16:27.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:26 smithi007 ceph-mon[32184]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T13:16:27.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:26 smithi007 ceph-mon[32184]: osdmap e70: 11 total, 10 up, 11 in 2024-03-23T13:16:27.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:26 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:27.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:26 smithi007 ceph-mon[32184]: from='osd.10 [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:16:27.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:26 smithi007 ceph-mon[32184]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-03-23T13:16:27.811 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T13:16:27.811 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T13:16:27.812 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-23T13:16:27.812 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-23T13:16:27.812 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125236 s, 837 MB/s 2024-03-23T13:16:27.812 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-23T13:16:28.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:27 smithi155 ceph-mon[36157]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:28.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:27 smithi155 ceph-mon[36157]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:16:28.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:27 smithi155 ceph-mon[36157]: osdmap e71: 11 total, 10 up, 11 in 2024-03-23T13:16:28.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:27 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:28.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:27 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:28.038 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:27 smithi130 ceph-mon[36225]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:28.038 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:27 smithi130 ceph-mon[36225]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:16:28.038 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:27 smithi130 ceph-mon[36225]: osdmap e71: 11 total, 10 up, 11 in 2024-03-23T13:16:28.038 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:27 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:28.038 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:27 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:28.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:27 smithi007 ceph-mon[32184]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-03-23T13:16:28.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:27 smithi007 ceph-mon[32184]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-03-23T13:16:28.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:27 smithi007 ceph-mon[32184]: osdmap e71: 11 total, 10 up, 11 in 2024-03-23T13:16:28.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:27 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:28.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:27 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:28.611 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: osd.10 [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] boot 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: osdmap e72: 11 total, 11 up, 11 in 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: osdmap e73: 11 total, 11 up, 11 in 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:28.829 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:28.830 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:28.830 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:28.830 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:28.830 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:28.830 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:28 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:28.856 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:16:29.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:16:29.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:16:29.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: osd.10 [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] boot 2024-03-23T13:16:29.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: osdmap e72: 11 total, 11 up, 11 in 2024-03-23T13:16:29.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: osdmap e73: 11 total, 11 up, 11 in 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:29.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:28 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:29.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:16:29.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:16:29.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: osd.10 [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] boot 2024-03-23T13:16:29.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: osdmap e72: 11 total, 11 up, 11 in 2024-03-23T13:16:29.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T13:16:29.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: osdmap e73: 11 total, 11 up, 11 in 2024-03-23T13:16:29.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:29.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:29.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:29.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:29.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:29.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:29.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:29.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:29.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:29.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:28 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:29.978 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:29 smithi155 ceph-mon[36157]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:29.978 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:29 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi130 2024-03-23T13:16:29.978 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:29 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi130 to 4270M 2024-03-23T13:16:29.979 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:29 smithi155 ceph-mon[36157]: osdmap e74: 11 total, 11 up, 11 in 2024-03-23T13:16:30.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:29 smithi007 ceph-mon[32184]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:30.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:29 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi130 2024-03-23T13:16:30.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:29 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi130 to 4270M 2024-03-23T13:16:30.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:29 smithi007 ceph-mon[32184]: osdmap e74: 11 total, 11 up, 11 in 2024-03-23T13:16:30.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:29 smithi130 ceph-mon[36225]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:30.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:29 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi130 2024-03-23T13:16:30.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:29 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi130 to 4270M 2024-03-23T13:16:30.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:29 smithi130 ceph-mon[36225]: osdmap e74: 11 total, 11 up, 11 in 2024-03-23T13:16:30.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.505+0000 7f6c1349a700 1 -- 172.21.15.155:0/474137461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c102330 msgr2=0x7f6c0c102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:30.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.505+0000 7f6c1349a700 1 --2- 172.21.15.155:0/474137461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c102330 0x7f6c0c102790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009990 tx=0x7f6bfc009ca0 comp rx=0 tx=0).stop 2024-03-23T13:16:30.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.505+0000 7f6c1349a700 1 -- 172.21.15.155:0/474137461 shutdown_connections 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.505+0000 7f6c1349a700 1 --2- 172.21.15.155:0/474137461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c0c102cd0 0x7f6c0c109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.505+0000 7f6c1349a700 1 --2- 172.21.15.155:0/474137461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c101130 0x7f6c0c101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.505+0000 7f6c1349a700 1 --2- 172.21.15.155:0/474137461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c102330 0x7f6c0c102790 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.505+0000 7f6c1349a700 1 -- 172.21.15.155:0/474137461 >> 172.21.15.155:0/474137461 conn(0x7f6c0c0fc6d0 msgr2=0x7f6c0c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.505+0000 7f6c1349a700 1 -- 172.21.15.155:0/474137461 shutdown_connections 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.505+0000 7f6c1349a700 1 -- 172.21.15.155:0/474137461 wait complete. 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c1349a700 1 Processor -- start 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c1349a700 1 -- start start 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c1349a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c0c101130 0x7f6c0c105000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c1349a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c102330 0x7f6c0c103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c1349a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c102cd0 0x7f6c0c103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c1349a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c104330 con 0x7f6c0c102cd0 2024-03-23T13:16:30.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c1349a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c1044a0 con 0x7f6c0c101130 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c1349a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c1a2300 con 0x7f6c0c102330 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c10a35700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c102330 0x7f6c0c103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c10a35700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c102330 0x7f6c0c103710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:45622/0 (socket says 172.21.15.155:45622) 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c10a35700 1 -- 172.21.15.155:0/441551402 learned_addr learned my addr 172.21.15.155:0/441551402 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c11236700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c0c101130 0x7f6c0c105000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c10a35700 1 -- 172.21.15.155:0/441551402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c0c101130 msgr2=0x7f6c0c105000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c10a35700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c0c101130 0x7f6c0c105000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c10a35700 1 -- 172.21.15.155:0/441551402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c102cd0 msgr2=0x7f6c0c103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c10a35700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c102cd0 0x7f6c0c103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c10a35700 1 -- 172.21.15.155:0/441551402 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc0095f0 con 0x7f6c0c102330 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.506+0000 7f6c10a35700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c102330 0x7f6c0c103710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0095c0 tx=0x7f6bfc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.507+0000 7f6c027fc700 1 -- 172.21.15.155:0/441551402 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc01d070 con 0x7f6c0c102330 2024-03-23T13:16:30.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.507+0000 7f6c1349a700 1 -- 172.21.15.155:0/441551402 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0c1a2500 con 0x7f6c0c102330 2024-03-23T13:16:30.508 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.507+0000 7f6c027fc700 1 -- 172.21.15.155:0/441551402 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bfc00fb40 con 0x7f6c0c102330 2024-03-23T13:16:30.508 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.507+0000 7f6c027fc700 1 -- 172.21.15.155:0/441551402 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0173f0 con 0x7f6c0c102330 2024-03-23T13:16:30.508 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.507+0000 7f6c1349a700 1 -- 172.21.15.155:0/441551402 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c0c1a2970 con 0x7f6c0c102330 2024-03-23T13:16:30.509 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.508+0000 7f6c1349a700 1 -- 172.21.15.155:0/441551402 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c0c068ec0 con 0x7f6c0c102330 2024-03-23T13:16:30.509 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.509+0000 7f6c027fc700 1 -- 172.21.15.155:0/441551402 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f6bfc00fcb0 con 0x7f6c0c102330 2024-03-23T13:16:30.509 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.510+0000 7f6c027fc700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6bf80a2150 0x7f6bf80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:30.509 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.510+0000 7f6c027fc700 1 -- 172.21.15.155:0/441551402 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f6bfc0ceae0 con 0x7f6c0c102330 2024-03-23T13:16:30.509 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.510+0000 7f6c11236700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6bf80a2150 0x7f6bf80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:30.509 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.510+0000 7f6c11236700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6bf80a2150 0x7f6bf80a4610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009550 tx=0x7f6c080091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:16:30.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.513+0000 7f6c027fc700 1 -- 172.21.15.155:0/441551402 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f6bfc05f280 con 0x7f6c0c102330 2024-03-23T13:16:30.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:30.717+0000 7f6c1349a700 1 -- 172.21.15.155:0/441551402 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6c0c0fe8b0 con 0x7f6bf80a2150 2024-03-23T13:16:31.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:31 smithi155 ceph-mon[36157]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:31.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:31 smithi155 ceph-mon[36157]: osdmap e75: 11 total, 11 up, 11 in 2024-03-23T13:16:31.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:31 smithi155 ceph-mon[36157]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:16:31.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:31 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:16:31.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:31 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:16:31.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:31 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:31.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:31 smithi007 ceph-mon[32184]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:31.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:31 smithi007 ceph-mon[32184]: osdmap e75: 11 total, 11 up, 11 in 2024-03-23T13:16:31.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:31 smithi007 ceph-mon[32184]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:16:31.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:31 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:16:31.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:31 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:16:31.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:31 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:31.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:31 smithi130 ceph-mon[36225]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:31.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:31 smithi130 ceph-mon[36225]: osdmap e75: 11 total, 11 up, 11 in 2024-03-23T13:16:31.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:31 smithi130 ceph-mon[36225]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:16:31.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:31 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T13:16:31.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:31 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T13:16:31.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:31 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:32.348 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:32 smithi155 ceph-mon[36157]: osdmap e76: 11 total, 11 up, 11 in 2024-03-23T13:16:32.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:32 smithi007 ceph-mon[32184]: osdmap e76: 11 total, 11 up, 11 in 2024-03-23T13:16:32.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:32 smithi130 ceph-mon[36225]: osdmap e76: 11 total, 11 up, 11 in 2024-03-23T13:16:33.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:33 smithi155 ceph-mon[36157]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:16:33.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:33 smithi007 ceph-mon[32184]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:16:33.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:33 smithi130 ceph-mon[36225]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T13:16:34.536 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:34 smithi155 ceph-mon[36157]: from='client.? 172.21.15.155:0/3401635848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34a0d3-024e-4b1f-907b-147a3fd1298d"}]: dispatch 2024-03-23T13:16:34.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:34 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34a0d3-024e-4b1f-907b-147a3fd1298d"}]: dispatch 2024-03-23T13:16:34.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:34 smithi155 ceph-mon[36157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc34a0d3-024e-4b1f-907b-147a3fd1298d"}]': finished 2024-03-23T13:16:34.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:34 smithi155 ceph-mon[36157]: osdmap e77: 12 total, 11 up, 12 in 2024-03-23T13:16:34.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:34 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:34.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:34 smithi155 ceph-mon[36157]: from='client.? 172.21.15.155:0/4244326784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:16:34.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:34 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/3401635848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34a0d3-024e-4b1f-907b-147a3fd1298d"}]: dispatch 2024-03-23T13:16:34.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:34 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34a0d3-024e-4b1f-907b-147a3fd1298d"}]: dispatch 2024-03-23T13:16:34.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:34 smithi007 ceph-mon[32184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc34a0d3-024e-4b1f-907b-147a3fd1298d"}]': finished 2024-03-23T13:16:34.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:34 smithi007 ceph-mon[32184]: osdmap e77: 12 total, 11 up, 12 in 2024-03-23T13:16:34.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:34 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:34.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:34 smithi007 ceph-mon[32184]: from='client.? 172.21.15.155:0/4244326784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:16:34.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:34 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/3401635848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34a0d3-024e-4b1f-907b-147a3fd1298d"}]: dispatch 2024-03-23T13:16:34.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:34 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34a0d3-024e-4b1f-907b-147a3fd1298d"}]: dispatch 2024-03-23T13:16:34.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:34 smithi130 ceph-mon[36225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc34a0d3-024e-4b1f-907b-147a3fd1298d"}]': finished 2024-03-23T13:16:34.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:34 smithi130 ceph-mon[36225]: osdmap e77: 12 total, 11 up, 12 in 2024-03-23T13:16:34.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:34 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:34.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:34 smithi130 ceph-mon[36225]: from='client.? 172.21.15.155:0/4244326784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T13:16:35.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:35 smithi155 ceph-mon[36157]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-03-23T13:16:35.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:35 smithi007 ceph-mon[32184]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-03-23T13:16:35.873 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:35 smithi130 ceph-mon[36225]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-03-23T13:16:37.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:37 smithi155 ceph-mon[36157]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T13:16:37.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:37 smithi007 ceph-mon[32184]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T13:16:37.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:37 smithi130 ceph-mon[36225]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T13:16:39.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:39 smithi155 ceph-mon[36157]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:39.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:39 smithi007 ceph-mon[32184]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:39.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:39 smithi130 ceph-mon[36225]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:41.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:41 smithi155 ceph-mon[36157]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:41.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T13:16:41.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:41.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:41 smithi007 ceph-mon[32184]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:41.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T13:16:41.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:41.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:41 smithi130 ceph-mon[36225]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:41.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:41 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T13:16:41.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:41 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:42.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:42 smithi155 ceph-mon[36157]: Deploying daemon osd.11 on smithi155 2024-03-23T13:16:42.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:42 smithi007 ceph-mon[32184]: Deploying daemon osd.11 on smithi155 2024-03-23T13:16:42.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:42 smithi130 ceph-mon[36225]: Deploying daemon osd.11 on smithi155 2024-03-23T13:16:43.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:43 smithi155 ceph-mon[36157]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:43.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:43 smithi007 ceph-mon[32184]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:43.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:43 smithi130 ceph-mon[36225]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:45.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:45 smithi155 ceph-mon[36157]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:45.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:45 smithi007 ceph-mon[32184]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:45.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:45 smithi130 ceph-mon[36225]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:47.803 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:47 smithi155 ceph-mon[36157]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:47.803 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:47 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:47.803 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:47 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:47.804 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:47 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:47.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:47 smithi007 ceph-mon[32184]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:47.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:47 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:47.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:47 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:47.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:47 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:47.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:47 smithi130 ceph-mon[36225]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:47.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:47 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:47.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:47 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:47.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:47 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:49.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:49 smithi155 ceph-mon[36157]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:49.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:49 smithi007 ceph-mon[32184]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:49.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:49 smithi130 ceph-mon[36225]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:50.533 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.534+0000 7f6c027fc700 1 -- 172.21.15.155:0/441551402 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6c0c0fe8b0 con 0x7f6bf80a2150 2024-03-23T13:16:50.533 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 11 on host 'smithi155' 2024-03-23T13:16:50.535 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 -- 172.21.15.155:0/441551402 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6bf80a2150 msgr2=0x7f6bf80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6bf80a2150 0x7f6bf80a4610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009550 tx=0x7f6c080091b0 comp rx=0 tx=0).stop 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 -- 172.21.15.155:0/441551402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c102330 msgr2=0x7f6c0c103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c102330 0x7f6c0c103710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0095c0 tx=0x7f6bfc00f690 comp rx=0 tx=0).stop 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 -- 172.21.15.155:0/441551402 shutdown_connections 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6bf80a2150 0x7f6bf80a4610 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c102cd0 0x7f6c0c103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c0c101130 0x7f6c0c105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 --2- 172.21.15.155:0/441551402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c102330 0x7f6c0c103710 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 -- 172.21.15.155:0/441551402 >> 172.21.15.155:0/441551402 conn(0x7f6c0c0fc6d0 msgr2=0x7f6c0c0fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 -- 172.21.15.155:0/441551402 shutdown_connections 2024-03-23T13:16:50.536 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:16:50.537+0000 7f6c1349a700 1 -- 172.21.15.155:0/441551402 wait complete. 2024-03-23T13:16:51.342 DEBUG:teuthology.orchestra.run.smithi155:osd.11> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.11.service 2024-03-23T13:16:51.344 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-23T13:16:51.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-23T13:16:51.606 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:16:51.727 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:51 smithi007 ceph-mon[32184]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:51.727 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:51 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:51.727 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:51 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:51.728 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:51 smithi007 ceph-mon[32184]: from='osd.11 [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T13:16:51.728 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:51 smithi007 ceph-mon[32184]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T13:16:51.786 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:51 smithi155 ceph-mon[36157]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:51.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:51 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:51.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:51 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:51.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:51 smithi155 ceph-mon[36157]: from='osd.11 [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T13:16:51.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:51 smithi155 ceph-mon[36157]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T13:16:51.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:51 smithi130 ceph-mon[36225]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:51.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:51 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:51.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:51 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:51.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:51 smithi130 ceph-mon[36225]: from='osd.11 [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T13:16:51.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:51 smithi130 ceph-mon[36225]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T13:16:52.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:52 smithi155 ceph-mon[36157]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T13:16:52.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:52 smithi155 ceph-mon[36157]: from='osd.11 [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:16:52.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:52 smithi155 ceph-mon[36157]: osdmap e78: 12 total, 11 up, 12 in 2024-03-23T13:16:52.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:52 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:52.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:52 smithi155 ceph-mon[36157]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:16:52.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:52 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:52.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:52 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:52.788 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:16:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:16:52.553+0000 7f4f8d059700 -1 osd.11 0 waiting for initial osdmap 2024-03-23T13:16:52.788 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:16:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:16:52.566+0000 7f4f88e4c700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T13:16:52.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:52 smithi007 ceph-mon[32184]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T13:16:52.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:52 smithi007 ceph-mon[32184]: from='osd.11 [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:16:52.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:52 smithi007 ceph-mon[32184]: osdmap e78: 12 total, 11 up, 12 in 2024-03-23T13:16:52.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:52 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:52.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:52 smithi007 ceph-mon[32184]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:16:52.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:52 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:52.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:52 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:52.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:52 smithi130 ceph-mon[36225]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T13:16:52.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:52 smithi130 ceph-mon[36225]: from='osd.11 [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:16:52.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:52 smithi130 ceph-mon[36225]: osdmap e78: 12 total, 11 up, 12 in 2024-03-23T13:16:52.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:52 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:52.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:52 smithi130 ceph-mon[36225]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-23T13:16:52.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:52 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:52.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:52 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.422+0000 7f22e472b700 1 -- 172.21.15.7:0/232737734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22dc102340 msgr2=0x7f22dc1027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.422+0000 7f22e472b700 1 --2- 172.21.15.7:0/232737734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22dc102340 0x7f22dc1027a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f22d00099c0 tx=0x7f22d0009cd0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.422+0000 7f22e472b700 1 -- 172.21.15.7:0/232737734 shutdown_connections 2024-03-23T13:16:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.422+0000 7f22e472b700 1 --2- 172.21.15.7:0/232737734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22dc102ce0 0x7f22dc109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.422+0000 7f22e472b700 1 --2- 172.21.15.7:0/232737734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22dc101140 0x7f22dc101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.422+0000 7f22e472b700 1 --2- 172.21.15.7:0/232737734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22dc102340 0x7f22dc1027a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.422+0000 7f22e472b700 1 -- 172.21.15.7:0/232737734 >> 172.21.15.7:0/232737734 conn(0x7f22dc0fc700 msgr2=0x7f22dc0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.422+0000 7f22e472b700 1 -- 172.21.15.7:0/232737734 shutdown_connections 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.422+0000 7f22e472b700 1 -- 172.21.15.7:0/232737734 wait complete. 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e472b700 1 Processor -- start 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e472b700 1 -- start start 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e472b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22dc101140 0x7f22dc1050d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e472b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22dc102340 0x7f22dc103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e472b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22dc102ce0 0x7f22dc103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e472b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1043e0 con 0x7f22dc102ce0 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e472b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc104550 con 0x7f22dc102340 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e472b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1a2250 con 0x7f22dc101140 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e2cc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22dc102ce0 0x7f22dc103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e2cc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22dc102ce0 0x7f22dc103c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60532/0 (socket says 172.21.15.7:60532) 2024-03-23T13:16:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e2cc8700 1 -- 172.21.15.7:0/1825714759 learned_addr learned my addr 172.21.15.7:0/1825714759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e1cc6700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22dc102340 0x7f22dc103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.423+0000 7f22e24c7700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22dc101140 0x7f22dc1050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22e2cc8700 1 -- 172.21.15.7:0/1825714759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22dc101140 msgr2=0x7f22dc1050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22e2cc8700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22dc101140 0x7f22dc1050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22e2cc8700 1 -- 172.21.15.7:0/1825714759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22dc102340 msgr2=0x7f22dc103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22e2cc8700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22dc102340 0x7f22dc103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22e2cc8700 1 -- 172.21.15.7:0/1825714759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d00095f0 con 0x7f22dc102ce0 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22e1cc6700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22dc102340 0x7f22dc103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22e2cc8700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22dc102ce0 0x7f22dc103c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22cc005fd0 tx=0x7f22cc00dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22db7fe700 1 -- 172.21.15.7:0/1825714759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc009930 con 0x7f22dc102ce0 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22db7fe700 1 -- 172.21.15.7:0/1825714759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22cc010460 con 0x7f22dc102ce0 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22db7fe700 1 -- 172.21.15.7:0/1825714759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00f620 con 0x7f22dc102ce0 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22e472b700 1 -- 172.21.15.7:0/1825714759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc1a2530 con 0x7f22dc102ce0 2024-03-23T13:16:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.424+0000 7f22e472b700 1 -- 172.21.15.7:0/1825714759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc1a2a30 con 0x7f22dc102ce0 2024-03-23T13:16:53.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.425+0000 7f22e472b700 1 -- 172.21.15.7:0/1825714759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc068ec0 con 0x7f22dc102ce0 2024-03-23T13:16:53.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.426+0000 7f22db7fe700 1 -- 172.21.15.7:0/1825714759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f22cc010980 con 0x7f22dc102ce0 2024-03-23T13:16:53.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.427+0000 7f22db7fe700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22c40a2150 0x7f22c40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:53.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.427+0000 7f22e24c7700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22c40a2150 0x7f22c40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:53.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.427+0000 7f22db7fe700 1 -- 172.21.15.7:0/1825714759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f22cc0cd5e0 con 0x7f22dc102ce0 2024-03-23T13:16:53.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.427+0000 7f22e24c7700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22c40a2150 0x7f22c40a4610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22dc1021a0 tx=0x7f22d4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:16:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.431+0000 7f22db7fe700 1 -- 172.21.15.7:0/1825714759 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f22cc096ad0 con 0x7f22dc102ce0 2024-03-23T13:16:53.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.636+0000 7f22e472b700 1 -- 172.21.15.7:0/1825714759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f22dc10ca80 con 0x7f22dc102ce0 2024-03-23T13:16:53.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.637+0000 7f22db7fe700 1 -- 172.21.15.7:0/1825714759 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f22cc09a190 con 0x7f22dc102ce0 2024-03-23T13:16:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:16:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 -- 172.21.15.7:0/1825714759 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22c40a2150 msgr2=0x7f22c40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22c40a2150 0x7f22c40a4610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22dc1021a0 tx=0x7f22d4009040 comp rx=0 tx=0).stop 2024-03-23T13:16:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 -- 172.21.15.7:0/1825714759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22dc102ce0 msgr2=0x7f22dc103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22dc102ce0 0x7f22dc103c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22cc005fd0 tx=0x7f22cc00dce0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 -- 172.21.15.7:0/1825714759 shutdown_connections 2024-03-23T13:16:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22c40a2150 0x7f22c40a4610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22dc102ce0 0x7f22dc103c60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22dc101140 0x7f22dc1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 --2- 172.21.15.7:0/1825714759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22dc102340 0x7f22dc103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 -- 172.21.15.7:0/1825714759 >> 172.21.15.7:0/1825714759 conn(0x7f22dc0fc700 msgr2=0x7f22dc0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:16:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 -- 172.21.15.7:0/1825714759 shutdown_connections 2024-03-23T13:16:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:53.639+0000 7f22e472b700 1 -- 172.21.15.7:0/1825714759 wait complete. 2024-03-23T13:16:53.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:53 smithi155 ceph-mon[36157]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:53.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:53 smithi155 ceph-mon[36157]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:16:53.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:53 smithi155 ceph-mon[36157]: osdmap e79: 12 total, 11 up, 12 in 2024-03-23T13:16:53.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:53.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:53 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:53.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:53 smithi007 ceph-mon[32184]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:53.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:53 smithi007 ceph-mon[32184]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:16:53.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:53 smithi007 ceph-mon[32184]: osdmap e79: 12 total, 11 up, 12 in 2024-03-23T13:16:53.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:53.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:53 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:53.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:53 smithi130 ceph-mon[36225]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:53.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:53 smithi130 ceph-mon[36225]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-23T13:16:53.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:53 smithi130 ceph-mon[36225]: osdmap e79: 12 total, 11 up, 12 in 2024-03-23T13:16:53.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:53.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:53 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:54.419 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1711199813,"num_in_osds":12,"osd_in_since":1711199793,"num_remapped_pgs":0} 2024-03-23T13:16:54.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T13:16:54.638 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:54 smithi155 ceph-mon[36157]: purged_snaps scrub starts 2024-03-23T13:16:54.638 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:54 smithi155 ceph-mon[36157]: purged_snaps scrub ok 2024-03-23T13:16:54.639 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:54 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:54.639 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:54 smithi155 ceph-mon[36157]: osd.11 [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] boot 2024-03-23T13:16:54.639 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:54 smithi155 ceph-mon[36157]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T13:16:54.639 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:54 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:54.639 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:54 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1825714759' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T13:16:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:16:54.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:54 smithi007 ceph-mon[32184]: purged_snaps scrub starts 2024-03-23T13:16:54.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:54 smithi007 ceph-mon[32184]: purged_snaps scrub ok 2024-03-23T13:16:54.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:54 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:54.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:54 smithi007 ceph-mon[32184]: osd.11 [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] boot 2024-03-23T13:16:54.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:54 smithi007 ceph-mon[32184]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T13:16:54.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:54 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:54.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:54 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1825714759' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T13:16:54.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:54 smithi130 ceph-mon[36225]: purged_snaps scrub starts 2024-03-23T13:16:54.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:54 smithi130 ceph-mon[36225]: purged_snaps scrub ok 2024-03-23T13:16:54.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:54 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:54.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:54 smithi130 ceph-mon[36225]: osd.11 [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] boot 2024-03-23T13:16:54.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:54 smithi130 ceph-mon[36225]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T13:16:54.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:54 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T13:16:54.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:54 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1825714759' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T13:16:55.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:55.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T13:16:55.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:55.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:55.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:55.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:55.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:55.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:55.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:55.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:55.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:55.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:55.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:55.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:55.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:55.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:55 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:55.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:55.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T13:16:55.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:55.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:55.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:55 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:56.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-03-23T13:16:56.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T13:16:56.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:56.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:56.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:56.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:56.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:56.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T13:16:56.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:56.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:56.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:56.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:56.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:16:56.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:16:56.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:16:56.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:55 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:16:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.248+0000 7f87e677e700 1 -- 172.21.15.7:0/598257529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e0101100 msgr2=0x7f87e0101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.248+0000 7f87e677e700 1 --2- 172.21.15.7:0/598257529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e0101100 0x7f87e0101560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009990 tx=0x7f87c8009ca0 comp rx=0 tx=0).stop 2024-03-23T13:16:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.248+0000 7f87e677e700 1 -- 172.21.15.7:0/598257529 shutdown_connections 2024-03-23T13:16:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.248+0000 7f87e677e700 1 --2- 172.21.15.7:0/598257529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101aa0 0x7f87e010df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.248+0000 7f87e677e700 1 --2- 172.21.15.7:0/598257529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87e0107140 0x7f87e0107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.248+0000 7f87e677e700 1 --2- 172.21.15.7:0/598257529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e0101100 0x7f87e0101560 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.248+0000 7f87e677e700 1 -- 172.21.15.7:0/598257529 >> 172.21.15.7:0/598257529 conn(0x7f87e0078030 msgr2=0x7f87e0078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.248+0000 7f87e677e700 1 -- 172.21.15.7:0/598257529 shutdown_connections 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.248+0000 7f87e677e700 1 -- 172.21.15.7:0/598257529 wait complete. 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87e677e700 1 Processor -- start 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87e677e700 1 -- start start 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87e677e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e0101100 0x7f87e0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87e677e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101aa0 0x7f87e0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87e677e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87e0107140 0x7f87e0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87e677e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01a6b70 con 0x7f87e0101100 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87e677e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01a6ce0 con 0x7f87e0101aa0 2024-03-23T13:16:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87e677e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01a6ea0 con 0x7f87e0107140 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e0101100 0x7f87e0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e0101100 0x7f87e0197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55540/0 (socket says 172.21.15.7:55540) 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.249+0000 7f87dffff700 1 -- 172.21.15.7:0/3101908688 learned_addr learned my addr 172.21.15.7:0/3101908688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87d7fff700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101aa0 0x7f87e0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87e4d1b700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87e0107140 0x7f87e0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87dffff700 1 -- 172.21.15.7:0/3101908688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87e0107140 msgr2=0x7f87e0197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87dffff700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87e0107140 0x7f87e0197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87dffff700 1 -- 172.21.15.7:0/3101908688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101aa0 msgr2=0x7f87e0197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87dffff700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101aa0 0x7f87e0197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87dffff700 1 -- 172.21.15.7:0/3101908688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c80095f0 con 0x7f87e0101100 2024-03-23T13:16:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87dffff700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e0101100 0x7f87e0197430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f87d000c920 tx=0x7f87d000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:16:56.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87ddffb700 1 -- 172.21.15.7:0/3101908688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0007b00 con 0x7f87e0101100 2024-03-23T13:16:56.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87ddffb700 1 -- 172.21.15.7:0/3101908688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d000ce70 con 0x7f87e0101100 2024-03-23T13:16:56.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87e677e700 1 -- 172.21.15.7:0/3101908688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e01a7180 con 0x7f87e0101100 2024-03-23T13:16:56.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87ddffb700 1 -- 172.21.15.7:0/3101908688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0005610 con 0x7f87e0101100 2024-03-23T13:16:56.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.250+0000 7f87e677e700 1 -- 172.21.15.7:0/3101908688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e01a7650 con 0x7f87e0101100 2024-03-23T13:16:56.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.251+0000 7f87e677e700 1 -- 172.21.15.7:0/3101908688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e0109660 con 0x7f87e0101100 2024-03-23T13:16:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.253+0000 7f87ddffb700 1 -- 172.21.15.7:0/3101908688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f87d0004750 con 0x7f87e0101100 2024-03-23T13:16:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.254+0000 7f87ddffb700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f87cc0a2150 0x7f87cc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.254+0000 7f87d7fff700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f87cc0a2150 0x7f87cc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.254+0000 7f87ddffb700 1 -- 172.21.15.7:0/3101908688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f87d00cd770 con 0x7f87e0101100 2024-03-23T13:16:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.254+0000 7f87d7fff700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f87cc0a2150 0x7f87cc0a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f87c800b5c0 tx=0x7f87c8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:16:56.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.257+0000 7f87ddffb700 1 -- 172.21.15.7:0/3101908688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f87d0096b40 con 0x7f87e0101100 2024-03-23T13:16:56.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.461+0000 7f87e677e700 1 -- 172.21.15.7:0/3101908688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87e0068ec0 con 0x7f87e0101100 2024-03-23T13:16:56.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.462+0000 7f87ddffb700 1 -- 172.21.15.7:0/3101908688 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18361 (secure 0 0 0) 0x7f87d009a200 con 0x7f87e0101100 2024-03-23T13:16:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:16:56.461 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:16:54.558135+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:16:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 -- 172.21.15.7:0/3101908688 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f87cc0a2150 msgr2=0x7f87cc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f87cc0a2150 0x7f87cc0a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f87c800b5c0 tx=0x7f87c8005960 comp rx=0 tx=0).stop 2024-03-23T13:16:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 -- 172.21.15.7:0/3101908688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e0101100 msgr2=0x7f87e0197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e0101100 0x7f87e0197430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f87d000c920 tx=0x7f87d000cc30 comp rx=0 tx=0).stop 2024-03-23T13:16:56.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 -- 172.21.15.7:0/3101908688 shutdown_connections 2024-03-23T13:16:56.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f87cc0a2150 0x7f87cc0a4610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:56.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87e0107140 0x7f87e0197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:56.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e0101100 0x7f87e0197430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:56.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 --2- 172.21.15.7:0/3101908688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87e0101aa0 0x7f87e0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:56.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 -- 172.21.15.7:0/3101908688 >> 172.21.15.7:0/3101908688 conn(0x7f87e0078030 msgr2=0x7f87e00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:16:56.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 -- 172.21.15.7:0/3101908688 shutdown_connections 2024-03-23T13:16:56.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:56.465+0000 7f87e677e700 1 -- 172.21.15.7:0/3101908688 wait complete. 2024-03-23T13:16:56.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:56 smithi007 ceph-mon[32184]: Detected new or changed devices on smithi155 2024-03-23T13:16:56.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:56 smithi007 ceph-mon[32184]: Adjusting osd_memory_target on smithi155 to 4270M 2024-03-23T13:16:56.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:56 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3101908688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:16:56.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:56 smithi130 ceph-mon[36225]: Detected new or changed devices on smithi155 2024-03-23T13:16:56.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:56 smithi130 ceph-mon[36225]: Adjusting osd_memory_target on smithi155 to 4270M 2024-03-23T13:16:56.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:56 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3101908688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:16:57.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:56 smithi155 ceph-mon[36157]: Detected new or changed devices on smithi155 2024-03-23T13:16:57.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:56 smithi155 ceph-mon[36157]: Adjusting osd_memory_target on smithi155 to 4270M 2024-03-23T13:16:57.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:56 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3101908688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:16:57.259 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T13:13:08.186719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T13:16:57.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-23T13:16:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:16:57.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:57 smithi007 ceph-mon[32184]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:16:57.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:57 smithi130 ceph-mon[36225]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:16:58.036 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:57 smithi155 ceph-mon[36157]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.163+0000 7f2d76990700 1 -- 172.21.15.7:0/2570837499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d70108230 msgr2=0x7f2d70108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.163+0000 7f2d76990700 1 --2- 172.21.15.7:0/2570837499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d70108230 0x7f2d70108610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2d600099c0 tx=0x7f2d60009cd0 comp rx=0 tx=0).stop 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.164+0000 7f2d76990700 1 -- 172.21.15.7:0/2570837499 shutdown_connections 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.164+0000 7f2d76990700 1 --2- 172.21.15.7:0/2570837499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d70102c10 0x7f2d7010f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.164+0000 7f2d76990700 1 --2- 172.21.15.7:0/2570837499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d70108230 0x7f2d70108610 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.164+0000 7f2d76990700 1 --2- 172.21.15.7:0/2570837499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d70102250 0x7f2d701026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.164+0000 7f2d76990700 1 -- 172.21.15.7:0/2570837499 >> 172.21.15.7:0/2570837499 conn(0x7f2d70079330 msgr2=0x7f2d70079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.164+0000 7f2d76990700 1 -- 172.21.15.7:0/2570837499 shutdown_connections 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.164+0000 7f2d76990700 1 -- 172.21.15.7:0/2570837499 wait complete. 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.164+0000 7f2d76990700 1 Processor -- start 2024-03-23T13:16:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.164+0000 7f2d76990700 1 -- start start 2024-03-23T13:16:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d76990700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d70102250 0x7f2d701986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d76990700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d70102c10 0x7f2d70198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d76990700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d70108230 0x7f2d7019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d76990700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7019d580 con 0x7f2d70108230 2024-03-23T13:16:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d76990700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7019d6f0 con 0x7f2d70102250 2024-03-23T13:16:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d76990700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7019d8b0 con 0x7f2d70102c10 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d74f2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d70108230 0x7f2d7019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d74f2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d70108230 0x7f2d7019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55560/0 (socket says 172.21.15.7:55560) 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d74f2d700 1 -- 172.21.15.7:0/656292926 learned_addr learned my addr 172.21.15.7:0/656292926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d6ffff700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d70102250 0x7f2d701986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d6f7fe700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d70102c10 0x7f2d70198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d74f2d700 1 -- 172.21.15.7:0/656292926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d70102c10 msgr2=0x7f2d70198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d74f2d700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d70102c10 0x7f2d70198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d74f2d700 1 -- 172.21.15.7:0/656292926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d70102250 msgr2=0x7f2d701986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d74f2d700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d70102250 0x7f2d701986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d74f2d700 1 -- 172.21.15.7:0/656292926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d600095f0 con 0x7f2d70108230 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.165+0000 7f2d74f2d700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d70108230 0x7f2d7019cf70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800b970 tx=0x7f2d6800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.166+0000 7f2d6d7fa700 1 -- 172.21.15.7:0/656292926 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800d860 con 0x7f2d70108230 2024-03-23T13:16:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.166+0000 7f2d76990700 1 -- 172.21.15.7:0/656292926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7019db90 con 0x7f2d70108230 2024-03-23T13:16:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.166+0000 7f2d6d7fa700 1 -- 172.21.15.7:0/656292926 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d6800e430 con 0x7f2d70108230 2024-03-23T13:16:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.166+0000 7f2d6d7fa700 1 -- 172.21.15.7:0/656292926 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800dea0 con 0x7f2d70108230 2024-03-23T13:16:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.166+0000 7f2d76990700 1 -- 172.21.15.7:0/656292926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d701a53f0 con 0x7f2d70108230 2024-03-23T13:16:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.167+0000 7f2d76990700 1 -- 172.21.15.7:0/656292926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7010a710 con 0x7f2d70108230 2024-03-23T13:16:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.168+0000 7f2d6d7fa700 1 -- 172.21.15.7:0/656292926 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f2d6800e910 con 0x7f2d70108230 2024-03-23T13:16:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.169+0000 7f2d6d7fa700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d580a21d0 0x7f2d580a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:16:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.169+0000 7f2d6ffff700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d580a21d0 0x7f2d580a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:16:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.169+0000 7f2d6d7fa700 1 -- 172.21.15.7:0/656292926 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2d68006e40 con 0x7f2d70108230 2024-03-23T13:16:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.169+0000 7f2d6ffff700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d580a21d0 0x7f2d580a4690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009990 tx=0x7f2d60005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:16:59.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.172+0000 7f2d6d7fa700 1 -- 172.21.15.7:0/656292926 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2d68096ac0 con 0x7f2d70108230 2024-03-23T13:16:59.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.374+0000 7f2d76990700 1 -- 172.21.15.7:0/656292926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2d70068ec0 con 0x7f2d70108230 2024-03-23T13:16:59.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.375+0000 7f2d6d7fa700 1 -- 172.21.15.7:0/656292926 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f2d6800e5a0 con 0x7f2d70108230 2024-03-23T13:16:59.373 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-23T13:16:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.376+0000 7f2d76990700 1 -- 172.21.15.7:0/656292926 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d580a21d0 msgr2=0x7f2d580a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.376+0000 7f2d76990700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d580a21d0 0x7f2d580a4690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009990 tx=0x7f2d60005a90 comp rx=0 tx=0).stop 2024-03-23T13:16:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.376+0000 7f2d76990700 1 -- 172.21.15.7:0/656292926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d70108230 msgr2=0x7f2d7019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:16:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.376+0000 7f2d76990700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d70108230 0x7f2d7019cf70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800b970 tx=0x7f2d6800bc80 comp rx=0 tx=0).stop 2024-03-23T13:16:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.377+0000 7f2d76990700 1 -- 172.21.15.7:0/656292926 shutdown_connections 2024-03-23T13:16:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.377+0000 7f2d76990700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d580a21d0 0x7f2d580a4690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.377+0000 7f2d76990700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d70108230 0x7f2d7019cf70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.377+0000 7f2d76990700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d70102250 0x7f2d701986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.377+0000 7f2d76990700 1 --2- 172.21.15.7:0/656292926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d70102c10 0x7f2d70198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:16:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.377+0000 7f2d76990700 1 -- 172.21.15.7:0/656292926 >> 172.21.15.7:0/656292926 conn(0x7f2d70079330 msgr2=0x7f2d70100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:16:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.377+0000 7f2d76990700 1 -- 172.21.15.7:0/656292926 shutdown_connections 2024-03-23T13:16:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:16:59.377+0000 7f2d76990700 1 -- 172.21.15.7:0/656292926 wait complete. 2024-03-23T13:16:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:59 smithi007 ceph-mon[32184]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:16:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:16:59 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/656292926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:16:59.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:59 smithi130 ceph-mon[36225]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:16:59.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:16:59 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/656292926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.a on smithi007 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.d on smithi007 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.g on smithi007 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.j on smithi007 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.b on smithi130 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.e on smithi130 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.h on smithi130 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.k on smithi130 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.c on smithi155 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.f on smithi155 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.i on smithi155 2024-03-23T13:16:59.977 INFO:tasks.cephadm:Adding mds.l on smithi155 2024-03-23T13:16:59.978 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l' 2024-03-23T13:17:00.010 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:59 smithi155 ceph-mon[36157]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:00.010 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:16:59 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/656292926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:17:00.230 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:17:01.676 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:01 smithi155 ceph-mon[36157]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:01.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:01 smithi130 ceph-mon[36225]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:02.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.931+0000 7f2c77734700 1 -- 172.21.15.155:0/3315249277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c70107f90 msgr2=0x7f2c7010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.931+0000 7f2c77734700 1 --2- 172.21.15.155:0/3315249277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c70107f90 0x7f2c7010a400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b0a0 tx=0x7f2c6800b3b0 comp rx=0 tx=0).stop 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.931+0000 7f2c77734700 1 -- 172.21.15.155:0/3315249277 shutdown_connections 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.931+0000 7f2c77734700 1 --2- 172.21.15.155:0/3315249277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c70107f90 0x7f2c7010a400 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.931+0000 7f2c77734700 1 --2- 172.21.15.155:0/3315249277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c70102d30 0x7f2c70105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.931+0000 7f2c77734700 1 --2- 172.21.15.155:0/3315249277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c70105660 0x7f2c70107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.931+0000 7f2c77734700 1 -- 172.21.15.155:0/3315249277 >> 172.21.15.155:0/3315249277 conn(0x7f2c700fc760 msgr2=0x7f2c700feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.931+0000 7f2c77734700 1 -- 172.21.15.155:0/3315249277 shutdown_connections 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.931+0000 7f2c77734700 1 -- 172.21.15.155:0/3315249277 wait complete. 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.932+0000 7f2c77734700 1 Processor -- start 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.932+0000 7f2c77734700 1 -- start start 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.932+0000 7f2c77734700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c70102d30 0x7f2c70198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.932+0000 7f2c77734700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c70105660 0x7f2c70198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.932+0000 7f2c77734700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c70107f90 0x7f2c7019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.932+0000 7f2c77734700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7019fdb0 con 0x7f2c70102d30 2024-03-23T13:17:02.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.932+0000 7f2c77734700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7019fef0 con 0x7f2c70105660 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.932+0000 7f2c77734700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c701a0080 con 0x7f2c70107f90 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.932+0000 7f2c75cd1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c70107f90 0x7f2c7019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c75cd1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c70107f90 0x7f2c7019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:47248/0 (socket says 172.21.15.155:47248) 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c754d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c70102d30 0x7f2c70198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c75cd1700 1 -- 172.21.15.155:0/3898109548 learned_addr learned my addr 172.21.15.155:0/3898109548 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c74ccf700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c70105660 0x7f2c70198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c75cd1700 1 -- 172.21.15.155:0/3898109548 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c70105660 msgr2=0x7f2c70198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c75cd1700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c70105660 0x7f2c70198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c75cd1700 1 -- 172.21.15.155:0/3898109548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c70102d30 msgr2=0x7f2c70198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c75cd1700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c70102d30 0x7f2c70198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c75cd1700 1 -- 172.21.15.155:0/3898109548 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c68009d00 con 0x7f2c70107f90 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c74ccf700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c70105660 0x7f2c70198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c75cd1700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c70107f90 0x7f2c7019f7e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c680094f0 tx=0x7f2c68009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:02.012 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c667fc700 1 -- 172.21.15.155:0/3898109548 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6800e040 con 0x7f2c70107f90 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c77734700 1 -- 172.21.15.155:0/3898109548 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c701a0300 con 0x7f2c70107f90 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c667fc700 1 -- 172.21.15.155:0/3898109548 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c68004e10 con 0x7f2c70107f90 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c667fc700 1 -- 172.21.15.155:0/3898109548 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680056e0 con 0x7f2c70107f90 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.933+0000 7f2c77734700 1 -- 172.21.15.155:0/3898109548 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c701a07c0 con 0x7f2c70107f90 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.934+0000 7f2c77734700 1 -- 172.21.15.155:0/3898109548 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c700fe2e0 con 0x7f2c70107f90 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.935+0000 7f2c667fc700 1 -- 172.21.15.155:0/3898109548 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f2c68005880 con 0x7f2c70107f90 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.936+0000 7f2c667fc700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2c5c0a2150 0x7f2c5c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.936+0000 7f2c667fc700 1 -- 172.21.15.155:0/3898109548 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2c680cf280 con 0x7f2c70107f90 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.936+0000 7f2c754d0700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2c5c0a2150 0x7f2c5c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.936+0000 7f2c754d0700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2c5c0a2150 0x7f2c5c0a4610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c005950 tx=0x7f2c6c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:02.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:01.940+0000 7f2c667fc700 1 -- 172.21.15.155:0/3898109548 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2c68098780 con 0x7f2c70107f90 2024-03-23T13:17:02.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:01 smithi007 ceph-mon[32184]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:02.144 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.146+0000 7f2c77734700 1 -- 172.21.15.155:0/3898109548 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}) -- 0x7f2c70062640 con 0x7f2c5c0a2150 2024-03-23T13:17:02.152 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.155+0000 7f2c667fc700 1 -- 172.21.15.155:0/3898109548 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f2c70062640 con 0x7f2c5c0a2150 2024-03-23T13:17:02.152 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mds.all update... 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 -- 172.21.15.155:0/3898109548 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2c5c0a2150 msgr2=0x7f2c5c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2c5c0a2150 0x7f2c5c0a4610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c005950 tx=0x7f2c6c00a300 comp rx=0 tx=0).stop 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 -- 172.21.15.155:0/3898109548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c70107f90 msgr2=0x7f2c7019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c70107f90 0x7f2c7019f7e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c680094f0 tx=0x7f2c68009520 comp rx=0 tx=0).stop 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 -- 172.21.15.155:0/3898109548 shutdown_connections 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2c5c0a2150 0x7f2c5c0a4610 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c70107f90 0x7f2c7019f7e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c70102d30 0x7f2c70198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 --2- 172.21.15.155:0/3898109548 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c70105660 0x7f2c70198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 -- 172.21.15.155:0/3898109548 >> 172.21.15.155:0/3898109548 conn(0x7f2c700fc760 msgr2=0x7f2c70103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:02.155 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 -- 172.21.15.155:0/3898109548 shutdown_connections 2024-03-23T13:17:02.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-23T13:17:02.157+0000 7f2c77734700 1 -- 172.21.15.155:0/3898109548 wait complete. 2024-03-23T13:17:02.857 DEBUG:teuthology.orchestra.run.smithi007:mds.a> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.a.service 2024-03-23T13:17:02.860 DEBUG:teuthology.orchestra.run.smithi007:mds.d> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.d.service 2024-03-23T13:17:02.904 DEBUG:teuthology.orchestra.run.smithi007:mds.g> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.g.service 2024-03-23T13:17:02.947 DEBUG:teuthology.orchestra.run.smithi007:mds.j> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.j.service 2024-03-23T13:17:02.990 DEBUG:teuthology.orchestra.run.smithi130:mds.b> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.b.service 2024-03-23T13:17:02.992 DEBUG:teuthology.orchestra.run.smithi130:mds.e> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.e.service 2024-03-23T13:17:03.035 DEBUG:teuthology.orchestra.run.smithi130:mds.h> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.h.service 2024-03-23T13:17:03.078 DEBUG:teuthology.orchestra.run.smithi130:mds.k> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.k.service 2024-03-23T13:17:03.121 DEBUG:teuthology.orchestra.run.smithi155:mds.c> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.c.service 2024-03-23T13:17:03.124 DEBUG:teuthology.orchestra.run.smithi155:mds.f> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.f.service 2024-03-23T13:17:03.166 DEBUG:teuthology.orchestra.run.smithi155:mds.i> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.i.service 2024-03-23T13:17:03.209 DEBUG:teuthology.orchestra.run.smithi155:mds.l> sudo journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.l.service 2024-03-23T13:17:03.252 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-23T13:17:03.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.374+0000 7f7837adf640 1 -- 172.21.15.7:0/85186419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7830102200 msgr2=0x7f7830108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.374+0000 7f7837adf640 1 --2- 172.21.15.7:0/85186419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7830102200 0x7f7830108a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7820009ca0 tx=0x7f782002c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.374+0000 7f7837adf640 1 -- 172.21.15.7:0/85186419 shutdown_connections 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.374+0000 7f7837adf640 1 --2- 172.21.15.7:0/85186419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7830102200 0x7f7830108a90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.374+0000 7f7837adf640 1 --2- 172.21.15.7:0/85186419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7830101840 0x7f7830101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.374+0000 7f7837adf640 1 --2- 172.21.15.7:0/85186419 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7830100650 0x7f7830100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.374+0000 7f7837adf640 1 -- 172.21.15.7:0/85186419 >> 172.21.15.7:0/85186419 conn(0x7f78300fbbc0 msgr2=0x7f78300fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.374+0000 7f7837adf640 1 -- 172.21.15.7:0/85186419 shutdown_connections 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.374+0000 7f7837adf640 1 -- 172.21.15.7:0/85186419 wait complete. 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f7837adf640 1 Processor -- start 2024-03-23T13:17:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f7837adf640 1 -- start start 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f7837adf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7830100650 0x7f7830197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f7837adf640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7830101840 0x7f7830197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f7837adf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7830102200 0x7f783019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f7837adf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f783019f2f0 con 0x7f7830102200 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f7837adf640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f783019f460 con 0x7f7830101840 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f7837adf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f783019f620 con 0x7f7830100650 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f783607c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7830102200 0x7f783019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f783607c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7830102200 0x7f783019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55564/0 (socket says 172.21.15.7:55564) 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f783607c640 1 -- 172.21.15.7:0/3859263251 learned_addr learned my addr 172.21.15.7:0/3859263251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f783587b640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7830100650 0x7f7830197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.375+0000 7f783507a640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7830101840 0x7f7830197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f783607c640 1 -- 172.21.15.7:0/3859263251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7830100650 msgr2=0x7f7830197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f783607c640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7830100650 0x7f7830197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f783607c640 1 -- 172.21.15.7:0/3859263251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7830101840 msgr2=0x7f7830197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f783607c640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7830101840 0x7f7830197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f783607c640 1 -- 172.21.15.7:0/3859263251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7820009900 con 0x7f7830102200 2024-03-23T13:17:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f783607c640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7830102200 0x7f783019edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7820037040 tx=0x7f7820034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f782effd640 1 -- 172.21.15.7:0/3859263251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820032eb0 con 0x7f7830102200 2024-03-23T13:17:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f782effd640 1 -- 172.21.15.7:0/3859263251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7820003e50 con 0x7f7830102200 2024-03-23T13:17:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f7837adf640 1 -- 172.21.15.7:0/3859263251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783019f8a0 con 0x7f7830102200 2024-03-23T13:17:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f782effd640 1 -- 172.21.15.7:0/3859263251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820040450 con 0x7f7830102200 2024-03-23T13:17:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.376+0000 7f7837adf640 1 -- 172.21.15.7:0/3859263251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783019fda0 con 0x7f7830102200 2024-03-23T13:17:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.377+0000 7f7837adf640 1 -- 172.21.15.7:0/3859263251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783010c060 con 0x7f7830102200 2024-03-23T13:17:03.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.378+0000 7f782effd640 1 -- 172.21.15.7:0/3859263251 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f782000a040 con 0x7f7830102200 2024-03-23T13:17:03.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.379+0000 7f782effd640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78180a2140 0x7f78180a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:03.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.379+0000 7f783587b640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78180a2140 0x7f78180a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:03.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.379+0000 7f782effd640 1 -- 172.21.15.7:0/3859263251 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f78200f0610 con 0x7f7830102200 2024-03-23T13:17:03.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.379+0000 7f783587b640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78180a2140 0x7f78180a4600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f78301016a0 tx=0x7f782800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:03.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.384+0000 7f782effd640 1 -- 172.21.15.7:0/3859263251 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f78200b9a80 con 0x7f7830102200 2024-03-23T13:17:03.425 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:03.425 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:03.425 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-03-23T13:17:03.425 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:03.425 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:17:03.425 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:03.425 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:17:03.425 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:03.425 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:03.426 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:03.426 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:03.426 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:03 smithi155 ceph-mon[36157]: Deploying daemon mds.l on smithi155 2024-03-23T13:17:03.443 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:03.443 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:03.443 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-03-23T13:17:03.443 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:03.443 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:17:03.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:03.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:17:03.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:03.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:03.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:03.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:03.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:03 smithi130 ceph-mon[36225]: Deploying daemon mds.l on smithi155 2024-03-23T13:17:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.568+0000 7f7837adf640 1 -- 172.21.15.7:0/3859263251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78301a0240 con 0x7f7830102200 2024-03-23T13:17:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.569+0000 7f782effd640 1 -- 172.21.15.7:0/3859263251 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18361 (secure 0 0 0) 0x7f78200bd140 con 0x7f7830102200 2024-03-23T13:17:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:03.568 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:16:54.558135+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:17:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 -- 172.21.15.7:0/3859263251 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78180a2140 msgr2=0x7f78180a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78180a2140 0x7f78180a4600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f78301016a0 tx=0x7f782800a2d0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 -- 172.21.15.7:0/3859263251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7830102200 msgr2=0x7f783019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7830102200 0x7f783019edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7820037040 tx=0x7f7820034d40 comp rx=0 tx=0).stop 2024-03-23T13:17:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 -- 172.21.15.7:0/3859263251 shutdown_connections 2024-03-23T13:17:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78180a2140 0x7f78180a4600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7830102200 0x7f783019edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7830101840 0x7f7830197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 --2- 172.21.15.7:0/3859263251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7830100650 0x7f7830197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 -- 172.21.15.7:0/3859263251 >> 172.21.15.7:0/3859263251 conn(0x7f78300fbbc0 msgr2=0x7f78300fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 -- 172.21.15.7:0/3859263251 shutdown_connections 2024-03-23T13:17:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.571+0000 7f7837adf640 1 -- 172.21.15.7:0/3859263251 wait complete. 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:03.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:03.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:03 smithi007 ceph-mon[32184]: Deploying daemon mds.l on smithi155 2024-03-23T13:17:03.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T13:13:08.186719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T13:17:03.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T13:17:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 -- 172.21.15.7:0/2784646521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b4101800 msgr2=0x7fb2b4101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 --2- 172.21.15.7:0/2784646521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b4101800 0x7fb2b4101c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009900 tx=0x7fb2a802bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 -- 172.21.15.7:0/2784646521 shutdown_connections 2024-03-23T13:17:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 --2- 172.21.15.7:0/2784646521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b41021c0 0x7fb2b4108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 --2- 172.21.15.7:0/2784646521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b4101800 0x7fb2b4101c80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 --2- 172.21.15.7:0/2784646521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2b4100610 0x7fb2b4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 -- 172.21.15.7:0/2784646521 >> 172.21.15.7:0/2784646521 conn(0x7fb2b40fbb80 msgr2=0x7fb2b40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 -- 172.21.15.7:0/2784646521 shutdown_connections 2024-03-23T13:17:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 -- 172.21.15.7:0/2784646521 wait complete. 2024-03-23T13:17:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 Processor -- start 2024-03-23T13:17:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.688+0000 7fb2ba384640 1 -- start start 2024-03-23T13:17:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2ba384640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b4100610 0x7fb2b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2ba384640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2b4101800 0x7fb2b419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2ba384640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b41021c0 0x7fb2b419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2ba384640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b419f2a0 con 0x7fb2b41021c0 2024-03-23T13:17:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2ba384640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b419f410 con 0x7fb2b4101800 2024-03-23T13:17:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2ba384640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b419f5d0 con 0x7fb2b4100610 2024-03-23T13:17:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b8921640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b41021c0 0x7fb2b419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b8921640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b41021c0 0x7fb2b419eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55578/0 (socket says 172.21.15.7:55578) 2024-03-23T13:17:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b8921640 1 -- 172.21.15.7:0/2980331103 learned_addr learned my addr 172.21.15.7:0/2980331103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b3fff640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b4100610 0x7fb2b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b37fe640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2b4101800 0x7fb2b419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b8921640 1 -- 172.21.15.7:0/2980331103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b4100610 msgr2=0x7fb2b40693e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b8921640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b4100610 0x7fb2b40693e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b8921640 1 -- 172.21.15.7:0/2980331103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2b4101800 msgr2=0x7fb2b419c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b8921640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2b4101800 0x7fb2b419c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b8921640 1 -- 172.21.15.7:0/2980331103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a4009490 con 0x7fb2b41021c0 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b8921640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b41021c0 0x7fb2b419eac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00cce0 tx=0x7fb2ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b17fa640 1 -- 172.21.15.7:0/2980331103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac00f450 con 0x7fb2b41021c0 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2ba384640 1 -- 172.21.15.7:0/2980331103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a8009560 con 0x7fb2b41021c0 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b17fa640 1 -- 172.21.15.7:0/2980331103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2ac00fa90 con 0x7fb2b41021c0 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2b17fa640 1 -- 172.21.15.7:0/2980331103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac018b90 con 0x7fb2b41021c0 2024-03-23T13:17:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.689+0000 7fb2ba384640 1 -- 172.21.15.7:0/2980331103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b419fb50 con 0x7fb2b41021c0 2024-03-23T13:17:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.691+0000 7fb2ba384640 1 -- 172.21.15.7:0/2980331103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b4107930 con 0x7fb2b41021c0 2024-03-23T13:17:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.692+0000 7fb2b17fa640 1 -- 172.21.15.7:0/2980331103 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fb2ac00fc00 con 0x7fb2b41021c0 2024-03-23T13:17:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.692+0000 7fb2b17fa640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb29c0a20c0 0x7fb29c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:03.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.692+0000 7fb2b3fff640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb29c0a20c0 0x7fb29c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:03.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.692+0000 7fb2b17fa640 1 -- 172.21.15.7:0/2980331103 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb2ac0cdf30 con 0x7fb2b41021c0 2024-03-23T13:17:03.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.693+0000 7fb2b3fff640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb29c0a20c0 0x7fb29c0a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4101660 tx=0x7fb2a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:03.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.697+0000 7fb2b17fa640 1 -- 172.21.15.7:0/2980331103 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb2ac0973a0 con 0x7fb2b41021c0 2024-03-23T13:17:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.876+0000 7fb2ba384640 1 -- 172.21.15.7:0/2980331103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2b410c270 con 0x7fb2b41021c0 2024-03-23T13:17:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.877+0000 7fb2b17fa640 1 -- 172.21.15.7:0/2980331103 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fb2ac09aa60 con 0x7fb2b41021c0 2024-03-23T13:17:03.876 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-23T13:17:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 -- 172.21.15.7:0/2980331103 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb29c0a20c0 msgr2=0x7fb29c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb29c0a20c0 0x7fb29c0a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4101660 tx=0x7fb2a4005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 -- 172.21.15.7:0/2980331103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b41021c0 msgr2=0x7fb2b419eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b41021c0 0x7fb2b419eac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00cce0 tx=0x7fb2ac00a620 comp rx=0 tx=0).stop 2024-03-23T13:17:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 -- 172.21.15.7:0/2980331103 shutdown_connections 2024-03-23T13:17:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb29c0a20c0 0x7fb29c0a4580 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b41021c0 0x7fb2b419eac0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2b4101800 0x7fb2b419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 --2- 172.21.15.7:0/2980331103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b4100610 0x7fb2b40693e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 -- 172.21.15.7:0/2980331103 >> 172.21.15.7:0/2980331103 conn(0x7fb2b40fbb80 msgr2=0x7fb2b4104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 -- 172.21.15.7:0/2980331103 shutdown_connections 2024-03-23T13:17:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:03.879+0000 7fb2ba384640 1 -- 172.21.15.7:0/2980331103 wait complete. 2024-03-23T13:17:03.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.003+0000 7f96d8a15640 1 -- 172.21.15.7:0/1936306027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d41021e0 msgr2=0x7f96d4108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.003+0000 7f96d8a15640 1 --2- 172.21.15.7:0/1936306027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d41021e0 0x7f96d4108a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009ca0 tx=0x7f96cc02c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.003+0000 7f96d8a15640 1 -- 172.21.15.7:0/1936306027 shutdown_connections 2024-03-23T13:17:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.003+0000 7f96d8a15640 1 --2- 172.21.15.7:0/1936306027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d41021e0 0x7f96d4108a70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.003+0000 7f96d8a15640 1 --2- 172.21.15.7:0/1936306027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96d4101820 0x7f96d4101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.003+0000 7f96d8a15640 1 --2- 172.21.15.7:0/1936306027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d4100630 0x7f96d4100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.003+0000 7f96d8a15640 1 -- 172.21.15.7:0/1936306027 >> 172.21.15.7:0/1936306027 conn(0x7f96d40fbbc0 msgr2=0x7f96d40fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.003+0000 7f96d8a15640 1 -- 172.21.15.7:0/1936306027 shutdown_connections 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.003+0000 7f96d8a15640 1 -- 172.21.15.7:0/1936306027 wait complete. 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d8a15640 1 Processor -- start 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d8a15640 1 -- start start 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d8a15640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d4100630 0x7f96d410adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d8a15640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96d4101820 0x7f96d41093f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d8a15640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d41021e0 0x7f96d4109930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d8a15640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96d410a090 con 0x7f96d4100630 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d8a15640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f96d410a200 con 0x7f96d4101820 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d8a15640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96d41a17c0 con 0x7f96d41021e0 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d4100630 0x7f96d410adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d4100630 0x7f96d410adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55586/0 (socket says 172.21.15.7:55586) 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d259c640 1 -- 172.21.15.7:0/3420339569 learned_addr learned my addr 172.21.15.7:0/3420339569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d2d9d640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d41021e0 0x7f96d4109930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96cbfff640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96d4101820 0x7f96d41093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d259c640 1 -- 172.21.15.7:0/3420339569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d41021e0 msgr2=0x7f96d4109930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d259c640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d41021e0 0x7f96d4109930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d259c640 1 -- 172.21.15.7:0/3420339569 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96d4101820 msgr2=0x7f96d41093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d259c640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96d4101820 0x7f96d41093f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.004+0000 7f96d259c640 1 -- 172.21.15.7:0/3420339569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc009900 con 0x7f96d4100630 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.005+0000 7f96d259c640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d4100630 0x7f96d410adb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f96c400e9f0 tx=0x7f96c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.005+0000 7f96cb7fe640 1 -- 172.21.15.7:0/3420339569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c40096e0 con 0x7f96d4100630 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.005+0000 7f96d8a15640 1 -- 172.21.15.7:0/3420339569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d41a1a70 con 0x7f96d4100630 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.005+0000 7f96cb7fe640 1 -- 172.21.15.7:0/3420339569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96c400cd80 con 0x7f96d4100630 2024-03-23T13:17:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.005+0000 7f96cb7fe640 1 -- 172.21.15.7:0/3420339569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4010430 con 0x7f96d4100630 2024-03-23T13:17:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.005+0000 7f96d8a15640 1 -- 172.21.15.7:0/3420339569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d41a1f30 con 0x7f96d4100630 2024-03-23T13:17:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.006+0000 7f96d8a15640 1 -- 172.21.15.7:0/3420339569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d4107950 con 0x7f96d4100630 2024-03-23T13:17:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.007+0000 7f96cb7fe640 1 -- 172.21.15.7:0/3420339569 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f96c4003680 con 0x7f96d4100630 2024-03-23T13:17:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.007+0000 7f96cb7fe640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f96c00a20c0 0x7f96c00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.007+0000 7f96cbfff640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f96c00a20c0 0x7f96c00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.007+0000 7f96cb7fe640 1 -- 172.21.15.7:0/3420339569 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f96c4014070 con 0x7f96d4100630 2024-03-23T13:17:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.008+0000 7f96cbfff640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f96c00a20c0 0x7f96c00a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f96d410ab20 tx=0x7f96bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.012+0000 7f96cb7fe640 1 -- 172.21.15.7:0/3420339569 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f96c40970b0 con 0x7f96d4100630 2024-03-23T13:17:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.233+0000 7f96d8a15640 1 -- 172.21.15.7:0/3420339569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96d410d140 con 0x7f96d4100630 2024-03-23T13:17:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.234+0000 7f96cb7fe640 1 -- 172.21.15.7:0/3420339569 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f96c409a770 con 0x7f96d4100630 2024-03-23T13:17:04.233 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:04.233 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-23T13:17:04.233 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 1 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 -- 172.21.15.7:0/3420339569 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f96c00a20c0 msgr2=0x7f96c00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f96c00a20c0 0x7f96c00a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f96d410ab20 tx=0x7f96bc005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 -- 172.21.15.7:0/3420339569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d4100630 msgr2=0x7f96d410adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d4100630 0x7f96d410adb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f96c400e9f0 tx=0x7f96c400eeb0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 -- 172.21.15.7:0/3420339569 shutdown_connections 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f96c00a20c0 0x7f96c00a4580 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d41021e0 0x7f96d4109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96d4101820 0x7f96d41093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 --2- 172.21.15.7:0/3420339569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d4100630 0x7f96d410adb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 -- 172.21.15.7:0/3420339569 >> 172.21.15.7:0/3420339569 conn(0x7f96d40fbbc0 msgr2=0x7f96d4104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 -- 172.21.15.7:0/3420339569 shutdown_connections 2024-03-23T13:17:04.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:04.236+0000 7f96d8a15640 1 -- 172.21.15.7:0/3420339569 wait complete. 2024-03-23T13:17:04.443 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:04 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3859263251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:04.443 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:04 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2980331103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:17:04.480 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:04 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3859263251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:04.480 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:04 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2980331103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:17:04.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:04 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3859263251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:04.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:04 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2980331103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:17:05.158 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:17:05 smithi155 podman[66210]: 2024-03-23 13:17:05.125489986 +0000 UTC m=+0.139952081 container create c3a25fe4bd6d9f8681c5ae94057969eff11f3e7dfcdd4c12b22aa3222e4d124a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:17:05.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:05 smithi007 ceph-mon[32184]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:05.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:05 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3420339569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:05.418 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:05 smithi155 ceph-mon[36157]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:05.418 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:05 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3420339569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:05.419 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:17:05 smithi155 podman[66210]: 2024-03-23 13:17:05.395110193 +0000 UTC m=+0.409572270 container init c3a25fe4bd6d9f8681c5ae94057969eff11f3e7dfcdd4c12b22aa3222e4d124a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:17:05.419 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:17:05 smithi155 podman[66210]: 2024-03-23 13:17:05.39783448 +0000 UTC m=+0.412296557 container start c3a25fe4bd6d9f8681c5ae94057969eff11f3e7dfcdd4c12b22aa3222e4d124a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-03-23T13:17:05.443 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:05 smithi130 ceph-mon[36225]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:05.443 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:05 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3420339569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:05.787 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:17:05 smithi155 ceph-mds[66229]: -- 172.21.15.155:0/2102825598 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564cb19861a0 con 0x564cb195c800 2024-03-23T13:17:05.787 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:17:05 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l[66225]: starting mds.l at 2024-03-23T13:17:05.787 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:17:05 smithi155 bash[66210]: c3a25fe4bd6d9f8681c5ae94057969eff11f3e7dfcdd4c12b22aa3222e4d124a 2024-03-23T13:17:05.787 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:17:05 smithi155 systemd[1]: Started Ceph mds.l for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:06.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.362+0000 7f28dcef4640 1 -- 172.21.15.7:0/17822541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d80ffdc0 msgr2=0x7f28d81001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.362+0000 7f28dcef4640 1 --2- 172.21.15.7:0/17822541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d80ffdc0 0x7f28d81001e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f28d000aa10 tx=0x7f28d002cf70 comp rx=0 tx=0).stop 2024-03-23T13:17:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.363+0000 7f28dcef4640 1 -- 172.21.15.7:0/17822541 shutdown_connections 2024-03-23T13:17:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.363+0000 7f28dcef4640 1 --2- 172.21.15.7:0/17822541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28d8101970 0x7f28d8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.363+0000 7f28dcef4640 1 --2- 172.21.15.7:0/17822541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28d8100fb0 0x7f28d8101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.363+0000 7f28dcef4640 1 --2- 172.21.15.7:0/17822541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d80ffdc0 0x7f28d81001e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.363+0000 7f28dcef4640 1 -- 172.21.15.7:0/17822541 >> 172.21.15.7:0/17822541 conn(0x7f28d80fb330 msgr2=0x7f28d80fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.363+0000 7f28dcef4640 1 -- 172.21.15.7:0/17822541 shutdown_connections 2024-03-23T13:17:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.363+0000 7f28dcef4640 1 -- 172.21.15.7:0/17822541 wait complete. 2024-03-23T13:17:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.363+0000 7f28dcef4640 1 Processor -- start 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.363+0000 7f28dcef4640 1 -- start start 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28dcef4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28d80ffdc0 0x7f28d8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28dcef4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28d8100fb0 0x7f28d8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28dcef4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8101970 0x7f28d819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28dcef4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f28d819f560 con 0x7f28d8101970 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28dcef4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f28d819f6d0 con 0x7f28d80ffdc0 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28dcef4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f28d819f890 con 0x7f28d8100fb0 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8101970 0x7f28d819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8101970 0x7f28d819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39682/0 (socket says 172.21.15.7:39682) 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d6d9d640 1 -- 172.21.15.7:0/694557521 learned_addr learned my addr 172.21.15.7:0/694557521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d659c640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28d80ffdc0 0x7f28d8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d5d9b640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28d8100fb0 0x7f28d8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d6d9d640 1 -- 172.21.15.7:0/694557521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28d8100fb0 msgr2=0x7f28d8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d6d9d640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28d8100fb0 0x7f28d8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d6d9d640 1 -- 172.21.15.7:0/694557521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28d80ffdc0 msgr2=0x7f28d8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d6d9d640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28d80ffdc0 0x7f28d8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d6d9d640 1 -- 172.21.15.7:0/694557521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d000a670 con 0x7f28d8101970 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28d6d9d640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8101970 0x7f28d819f020 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f28c800b970 tx=0x7f28c800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28cf7fe640 1 -- 172.21.15.7:0/694557521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c800cb40 con 0x7f28d8101970 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28cf7fe640 1 -- 172.21.15.7:0/694557521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c800cca0 con 0x7f28d8101970 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28dcef4640 1 -- 172.21.15.7:0/694557521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d819fb70 con 0x7f28d8101970 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28cf7fe640 1 -- 172.21.15.7:0/694557521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c80077d0 con 0x7f28d8101970 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.364+0000 7f28dcef4640 1 -- 172.21.15.7:0/694557521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d81a0080 con 0x7f28d8101970 2024-03-23T13:17:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.366+0000 7f28dcef4640 1 -- 172.21.15.7:0/694557521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d8107160 con 0x7f28d8101970 2024-03-23T13:17:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.367+0000 7f28cf7fe640 1 -- 172.21.15.7:0/694557521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f28c8014440 con 0x7f28d8101970 2024-03-23T13:17:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.367+0000 7f28cf7fe640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f28c40a2140 0x7f28c40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.367+0000 7f28cf7fe640 1 -- 172.21.15.7:0/694557521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f28c80cd010 con 0x7f28d8101970 2024-03-23T13:17:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.367+0000 7f28d659c640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f28c40a2140 0x7f28c40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.368+0000 7f28d659c640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f28c40a2140 0x7f28c40a4600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f28d0000c00 tx=0x7f28d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:06.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.372+0000 7f28cf7fe640 1 -- 172.21.15.7:0/694557521 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f28c8096530 con 0x7f28d8101970 2024-03-23T13:17:06.426 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:06 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6832/1319723529,v1:172.21.15.155:6833/1319723529] up:boot 2024-03-23T13:17:06.426 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:06 smithi130 ceph-mon[36225]: fsmap 1 up:standby 2024-03-23T13:17:06.426 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T13:17:06.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:06 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6832/1319723529,v1:172.21.15.155:6833/1319723529] up:boot 2024-03-23T13:17:06.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:06 smithi007 ceph-mon[32184]: fsmap 1 up:standby 2024-03-23T13:17:06.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T13:17:06.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:06.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:06.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:06.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:06.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:06.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:06.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:06 smithi007 ceph-mon[32184]: Deploying daemon mds.k on smithi130 2024-03-23T13:17:06.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.592+0000 7f28dcef4640 1 -- 172.21.15.7:0/694557521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28d810b260 con 0x7f28d8101970 2024-03-23T13:17:06.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.593+0000 7f28cf7fe640 1 -- 172.21.15.7:0/694557521 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f28c8099bf0 con 0x7f28d8101970 2024-03-23T13:17:06.592 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:06.592 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T13:17:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-03-23T13:17:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 -- 172.21.15.7:0/694557521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f28c40a2140 msgr2=0x7f28c40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f28c40a2140 0x7f28c40a4600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f28d0000c00 tx=0x7f28d0005fd0 comp rx=0 tx=0).stop 2024-03-23T13:17:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 -- 172.21.15.7:0/694557521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8101970 msgr2=0x7f28d819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8101970 0x7f28d819f020 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f28c800b970 tx=0x7f28c800be30 comp rx=0 tx=0).stop 2024-03-23T13:17:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 -- 172.21.15.7:0/694557521 shutdown_connections 2024-03-23T13:17:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f28c40a2140 0x7f28c40a4600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8101970 0x7f28d819f020 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28d8100fb0 0x7f28d8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:06.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 --2- 172.21.15.7:0/694557521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28d80ffdc0 0x7f28d8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:06.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 -- 172.21.15.7:0/694557521 >> 172.21.15.7:0/694557521 conn(0x7f28d80fb330 msgr2=0x7f28d8104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:06.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 -- 172.21.15.7:0/694557521 shutdown_connections 2024-03-23T13:17:06.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:06.595+0000 7f28dcef4640 1 -- 172.21.15.7:0/694557521 wait complete. 2024-03-23T13:17:06.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:06.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:06.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:06.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:06.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:06.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:06.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:06 smithi130 ceph-mon[36225]: Deploying daemon mds.k on smithi130 2024-03-23T13:17:06.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:06 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6832/1319723529,v1:172.21.15.155:6833/1319723529] up:boot 2024-03-23T13:17:06.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:06 smithi155 ceph-mon[36157]: fsmap 1 up:standby 2024-03-23T13:17:06.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T13:17:06.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:06.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:06.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:06.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:06.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:06.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:06.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:06 smithi155 ceph-mon[36157]: Deploying daemon mds.k on smithi130 2024-03-23T13:17:07.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:07 smithi007 ceph-mon[32184]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:07.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:07 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/694557521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:07.928 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:07 smithi130 ceph-mon[36225]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:07.929 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:07 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/694557521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:07.929 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:17:07 smithi130 systemd[1]: Starting Ceph mds.k for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:17:08.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:07 smithi155 ceph-mon[36157]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:08.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:07 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/694557521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:08.193 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:17:07 smithi130 podman[66068]: 2024-03-23 13:17:07.833527698 +0000 UTC m=+0.017751632 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:17:08.193 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:17:07 smithi130 podman[66068]: 2024-03-23 13:17:07.977349455 +0000 UTC m=+0.161573398 container create 0c546a506d7eff7d991eaaa0036f46a923d761ab2f47a4badb40cb04bac60c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True) 2024-03-23T13:17:08.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:08.693 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:17:08 smithi130 podman[66068]: 2024-03-23 13:17:08.213844449 +0000 UTC m=+0.398068343 container init 0c546a506d7eff7d991eaaa0036f46a923d761ab2f47a4badb40cb04bac60c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-23T13:17:08.693 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:17:08 smithi130 podman[66068]: 2024-03-23 13:17:08.216710015 +0000 UTC m=+0.400933907 container start 0c546a506d7eff7d991eaaa0036f46a923d761ab2f47a4badb40cb04bac60c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:17:08.694 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:17:08 smithi130 ceph-mds[66087]: -- 172.21.15.130:0/1846468990 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558cc37da1a0 con 0x558cc37b1000 2024-03-23T13:17:08.694 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:17:08 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k[66083]: starting mds.k at 2024-03-23T13:17:08.694 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:17:08 smithi130 bash[66068]: 0c546a506d7eff7d991eaaa0036f46a923d761ab2f47a4badb40cb04bac60c10 2024-03-23T13:17:08.694 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:17:08 smithi130 systemd[1]: Started Ceph mds.k for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.711+0000 7f1e9f00b640 1 -- 172.21.15.7:0/68918283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98102950 msgr2=0x7f1e98102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.711+0000 7f1e9f00b640 1 --2- 172.21.15.7:0/68918283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98102950 0x7f1e98102dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009900 tx=0x7f1e8402bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.711+0000 7f1e9f00b640 1 -- 172.21.15.7:0/68918283 shutdown_connections 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.711+0000 7f1e9f00b640 1 --2- 172.21.15.7:0/68918283 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e98103310 0x7f1e98109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.711+0000 7f1e9f00b640 1 --2- 172.21.15.7:0/68918283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98102950 0x7f1e98102dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.711+0000 7f1e9f00b640 1 --2- 172.21.15.7:0/68918283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e98101760 0x7f1e98101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.711+0000 7f1e9f00b640 1 -- 172.21.15.7:0/68918283 >> 172.21.15.7:0/68918283 conn(0x7f1e980fccb0 msgr2=0x7f1e980ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.711+0000 7f1e9f00b640 1 -- 172.21.15.7:0/68918283 shutdown_connections 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.711+0000 7f1e9f00b640 1 -- 172.21.15.7:0/68918283 wait complete. 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9f00b640 1 Processor -- start 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9f00b640 1 -- start start 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9f00b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e98101760 0x7f1e980754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9f00b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e98102950 0x7f1e98073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9f00b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98103310 0x7f1e98074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9f00b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e980747a0 con 0x7f1e98103310 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9f00b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1e98074910 con 0x7f1e98102950 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9f00b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981a18c0 con 0x7f1e98101760 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9d5a8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98103310 0x7f1e98074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9d5a8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98103310 0x7f1e98074040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39686/0 (socket says 172.21.15.7:39686) 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9d5a8640 1 -- 172.21.15.7:0/1451136661 learned_addr learned my addr 172.21.15.7:0/1451136661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9cda7640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e98101760 0x7f1e980754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e97fff640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e98102950 0x7f1e98073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9d5a8640 1 -- 172.21.15.7:0/1451136661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e98101760 msgr2=0x7f1e980754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9d5a8640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e98101760 0x7f1e980754c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9d5a8640 1 -- 172.21.15.7:0/1451136661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e98102950 msgr2=0x7f1e98073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9d5a8640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e98102950 0x7f1e98073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9d5a8640 1 -- 172.21.15.7:0/1451136661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e84009560 con 0x7f1e98103310 2024-03-23T13:17:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e9d5a8640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98103310 0x7f1e98074040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000cd10 tx=0x7f1e9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.712+0000 7f1e95ffb640 1 -- 172.21.15.7:0/1451136661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000f450 con 0x7f1e98103310 2024-03-23T13:17:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.713+0000 7f1e9f00b640 1 -- 172.21.15.7:0/1451136661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e981a1b70 con 0x7f1e98103310 2024-03-23T13:17:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.713+0000 7f1e95ffb640 1 -- 172.21.15.7:0/1451136661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e9000fa90 con 0x7f1e98103310 2024-03-23T13:17:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.713+0000 7f1e95ffb640 1 -- 172.21.15.7:0/1451136661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90018b70 con 0x7f1e98103310 2024-03-23T13:17:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.713+0000 7f1e9f00b640 1 -- 172.21.15.7:0/1451136661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e981a2030 con 0x7f1e98103310 2024-03-23T13:17:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.714+0000 7f1e9f00b640 1 -- 172.21.15.7:0/1451136661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e98108a80 con 0x7f1e98103310 2024-03-23T13:17:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.715+0000 7f1e95ffb640 1 -- 172.21.15.7:0/1451136661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f1e9000fc00 con 0x7f1e98103310 2024-03-23T13:17:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.715+0000 7f1e95ffb640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1e880a20c0 0x7f1e880a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.715+0000 7f1e9cda7640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1e880a20c0 0x7f1e880a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.715+0000 7f1e95ffb640 1 -- 172.21.15.7:0/1451136661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1e900cdca0 con 0x7f1e98103310 2024-03-23T13:17:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.716+0000 7f1e9cda7640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1e880a20c0 0x7f1e880a4580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1e981027b0 tx=0x7f1e8c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.720+0000 7f1e95ffb640 1 -- 172.21.15.7:0/1451136661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1e90097110 con 0x7f1e98103310 2024-03-23T13:17:08.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.949+0000 7f1e9f00b640 1 -- 172.21.15.7:0/1451136661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e9810d3e0 con 0x7f1e98103310 2024-03-23T13:17:08.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.950+0000 7f1e95ffb640 1 -- 172.21.15.7:0/1451136661 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f1e9009a7d0 con 0x7f1e98103310 2024-03-23T13:17:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:08.949 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T13:17:08.949 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-03-23T13:17:08.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.951+0000 7f1e9f00b640 1 -- 172.21.15.7:0/1451136661 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1e880a20c0 msgr2=0x7f1e880a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.951+0000 7f1e9f00b640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1e880a20c0 0x7f1e880a4580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1e981027b0 tx=0x7f1e8c008040 comp rx=0 tx=0).stop 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.951+0000 7f1e9f00b640 1 -- 172.21.15.7:0/1451136661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98103310 msgr2=0x7f1e98074040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.951+0000 7f1e9f00b640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98103310 0x7f1e98074040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000cd10 tx=0x7f1e9000a620 comp rx=0 tx=0).stop 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.952+0000 7f1e9f00b640 1 -- 172.21.15.7:0/1451136661 shutdown_connections 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.952+0000 7f1e9f00b640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1e880a20c0 0x7f1e880a4580 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.952+0000 7f1e9f00b640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98103310 0x7f1e98074040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.952+0000 7f1e9f00b640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e98102950 0x7f1e98073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.952+0000 7f1e9f00b640 1 --2- 172.21.15.7:0/1451136661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e98101760 0x7f1e980754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.952+0000 7f1e9f00b640 1 -- 172.21.15.7:0/1451136661 >> 172.21.15.7:0/1451136661 conn(0x7f1e980fccb0 msgr2=0x7f1e98105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.952+0000 7f1e9f00b640 1 -- 172.21.15.7:0/1451136661 shutdown_connections 2024-03-23T13:17:08.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:08.952+0000 7f1e9f00b640 1 -- 172.21.15.7:0/1451136661 wait complete. 2024-03-23T13:17:09.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:09.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:09.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:09.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:09.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:09.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:09.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:09.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: Deploying daemon mds.j on smithi007 2024-03-23T13:17:09.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.130:6832/3818242586,v1:172.21.15.130:6833/3818242586] up:boot 2024-03-23T13:17:09.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: fsmap 2 up:standby 2024-03-23T13:17:09.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T13:17:09.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:09 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1451136661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:09.618 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:09.618 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:09.618 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:09.618 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:09.618 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:09.618 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:09.618 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:09.618 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: Deploying daemon mds.j on smithi007 2024-03-23T13:17:09.618 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.130:6832/3818242586,v1:172.21.15.130:6833/3818242586] up:boot 2024-03-23T13:17:09.619 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: fsmap 2 up:standby 2024-03-23T13:17:09.619 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T13:17:09.619 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:09 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1451136661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:09.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:09.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:09.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:09.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:09.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:09.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:09.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:09.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: Deploying daemon mds.j on smithi007 2024-03-23T13:17:09.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.130:6832/3818242586,v1:172.21.15.130:6833/3818242586] up:boot 2024-03-23T13:17:09.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: fsmap 2 up:standby 2024-03-23T13:17:09.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T13:17:09.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:09 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1451136661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:10.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:10.988 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:17:10 smithi007 podman[72395]: 2024-03-23 13:17:10.963783972 +0000 UTC m=+0.132302419 container create 287fc6b1b0f54ddda2fb3d0cd5a60de422d1b188198f5de8a9e557c9a0895698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-j, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-23T13:17:11.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.070+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/2869876610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b41021d0 msgr2=0x7fa6b41045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:11.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.070+0000 7fa6bc5fd640 1 --2- 172.21.15.7:0/2869876610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b41021d0 0x7fa6b41045c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009900 tx=0x7fa6b002bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:11.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/2869876610 shutdown_connections 2024-03-23T13:17:11.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 --2- 172.21.15.7:0/2869876610 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b4107430 0x7fa6b4109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:11.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 --2- 172.21.15.7:0/2869876610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b4104b00 0x7fa6b4106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:11.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 --2- 172.21.15.7:0/2869876610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b41021d0 0x7fa6b41045c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:11.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/2869876610 >> 172.21.15.7:0/2869876610 conn(0x7fa6b40fbbc0 msgr2=0x7fa6b40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/2869876610 shutdown_connections 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/2869876610 wait complete. 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 Processor -- start 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 -- start start 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b41021d0 0x7fa6b4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b4104b00 0x7fa6b4197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b4107430 0x7fa6b419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b419f340 con 0x7fa6b4104b00 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b419f4b0 con 0x7fa6b41021d0 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6bc5fd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b419f670 con 0x7fa6b4107430 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.071+0000 7fa6b9b98640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b4104b00 0x7fa6b4197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6b9b98640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b4104b00 0x7fa6b4197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39718/0 (socket says 172.21.15.7:39718) 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6ba399640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b41021d0 0x7fa6b4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6b9b98640 1 -- 172.21.15.7:0/3299862953 learned_addr learned my addr 172.21.15.7:0/3299862953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6bab9a640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b4107430 0x7fa6b419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6b9b98640 1 -- 172.21.15.7:0/3299862953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b4107430 msgr2=0x7fa6b419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6b9b98640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b4107430 0x7fa6b419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6b9b98640 1 -- 172.21.15.7:0/3299862953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b41021d0 msgr2=0x7fa6b4197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6b9b98640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b41021d0 0x7fa6b4197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6b9b98640 1 -- 172.21.15.7:0/3299862953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b0009560 con 0x7fa6b4104b00 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6b9b98640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b4104b00 0x7fa6b4197d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c970 tx=0x7fa6a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6ab7fe640 1 -- 172.21.15.7:0/3299862953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4007bb0 con 0x7fa6b4104b00 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6ab7fe640 1 -- 172.21.15.7:0/3299862953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4007d10 con 0x7fa6b4104b00 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/3299862953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b419f950 con 0x7fa6b4104b00 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6ab7fe640 1 -- 172.21.15.7:0/3299862953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4018730 con 0x7fa6b4104b00 2024-03-23T13:17:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.072+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/3299862953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b419fe10 con 0x7fa6b4104b00 2024-03-23T13:17:11.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.073+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/3299862953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b40fd6b0 con 0x7fa6b4104b00 2024-03-23T13:17:11.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.074+0000 7fa6ab7fe640 1 -- 172.21.15.7:0/3299862953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fa6a400f450 con 0x7fa6b4104b00 2024-03-23T13:17:11.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.075+0000 7fa6ab7fe640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa6a00a20c0 0x7fa6a00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:11.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.075+0000 7fa6ba399640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa6a00a20c0 0x7fa6a00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:11.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.075+0000 7fa6ab7fe640 1 -- 172.21.15.7:0/3299862953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa6a40cd870 con 0x7fa6b4104b00 2024-03-23T13:17:11.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.075+0000 7fa6ba399640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa6a00a20c0 0x7fa6a00a4580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0006010 tx=0x7fa6b0005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:11.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.079+0000 7fa6ab7fe640 1 -- 172.21.15.7:0/3299862953 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa6a4096ce0 con 0x7fa6b4104b00 2024-03-23T13:17:11.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.298+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/3299862953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6b410d310 con 0x7fa6b4104b00 2024-03-23T13:17:11.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.299+0000 7fa6ab7fe640 1 -- 172.21.15.7:0/3299862953 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fa6a409a3a0 con 0x7fa6b4104b00 2024-03-23T13:17:11.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:11.298 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T13:17:11.298 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/3299862953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa6a00a20c0 msgr2=0x7fa6a00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa6a00a20c0 0x7fa6a00a4580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0006010 tx=0x7fa6b0005d90 comp rx=0 tx=0).stop 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/3299862953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b4104b00 msgr2=0x7fa6b4197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b4104b00 0x7fa6b4197d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c970 tx=0x7fa6a400ce30 comp rx=0 tx=0).stop 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/3299862953 shutdown_connections 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa6a00a20c0 0x7fa6a00a4580 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b4107430 0x7fa6b419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b4104b00 0x7fa6b4197d90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 --2- 172.21.15.7:0/3299862953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b41021d0 0x7fa6b4197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/3299862953 >> 172.21.15.7:0/3299862953 conn(0x7fa6b40fbbc0 msgr2=0x7fa6b4102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/3299862953 shutdown_connections 2024-03-23T13:17:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:11.301+0000 7fa6bc5fd640 1 -- 172.21.15.7:0/3299862953 wait complete. 2024-03-23T13:17:11.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:11 smithi007 ceph-mon[32184]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:11.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:11 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3299862953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:11.576 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:17:11 smithi007 podman[72395]: 2024-03-23 13:17:11.375374451 +0000 UTC m=+0.543892905 container init 287fc6b1b0f54ddda2fb3d0cd5a60de422d1b188198f5de8a9e557c9a0895698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-j, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:17:11.576 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:17:11 smithi007 podman[72395]: 2024-03-23 13:17:11.378349975 +0000 UTC m=+0.546868436 container start 287fc6b1b0f54ddda2fb3d0cd5a60de422d1b188198f5de8a9e557c9a0895698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-23T13:17:11.576 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:17:11 smithi007 ceph-mds[72455]: -- 172.21.15.7:0/2789107431 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5619cb1b41a0 con 0x5619cb18a800 2024-03-23T13:17:11.576 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:17:11 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-j[72449]: starting mds.j at 2024-03-23T13:17:11.576 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:17:11 smithi007 bash[72395]: 287fc6b1b0f54ddda2fb3d0cd5a60de422d1b188198f5de8a9e557c9a0895698 2024-03-23T13:17:11.576 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:17:11 smithi007 systemd[1]: Started Ceph mds.j for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:11.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:11 smithi130 ceph-mon[36225]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:11.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:11 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3299862953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:11.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:11 smithi155 ceph-mon[36157]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:11.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:11 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3299862953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:12.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:12.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:12.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:12.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:12.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:12.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:12.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:12 smithi155 ceph-mon[36157]: Deploying daemon mds.d on smithi007 2024-03-23T13:17:12.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:12 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.7:6834/3229137818,v1:172.21.15.7:6835/3229137818] up:boot 2024-03-23T13:17:12.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:12 smithi155 ceph-mon[36157]: fsmap 3 up:standby 2024-03-23T13:17:12.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:12 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T13:17:12.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:12.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:12.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:12.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:12.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:12.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:12.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:12 smithi007 ceph-mon[32184]: Deploying daemon mds.d on smithi007 2024-03-23T13:17:12.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:12 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.7:6834/3229137818,v1:172.21.15.7:6835/3229137818] up:boot 2024-03-23T13:17:12.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:12 smithi007 ceph-mon[32184]: fsmap 3 up:standby 2024-03-23T13:17:12.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:12 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T13:17:12.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:12.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:12.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:12.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:12.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:12.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:12.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:12 smithi130 ceph-mon[36225]: Deploying daemon mds.d on smithi007 2024-03-23T13:17:12.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:12 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.7:6834/3229137818,v1:172.21.15.7:6835/3229137818] up:boot 2024-03-23T13:17:12.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:12 smithi130 ceph-mon[36225]: fsmap 3 up:standby 2024-03-23T13:17:12.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:12 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T13:17:13.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.440+0000 7ff955d80640 1 -- 172.21.15.7:0/1304922607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9501033b0 msgr2=0x7ff950109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.440+0000 7ff955d80640 1 --2- 172.21.15.7:0/1304922607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9501033b0 0x7ff950109c40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff948009ca0 tx=0x7ff94802c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.440+0000 7ff955d80640 1 -- 172.21.15.7:0/1304922607 shutdown_connections 2024-03-23T13:17:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.440+0000 7ff955d80640 1 --2- 172.21.15.7:0/1304922607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9501033b0 0x7ff950109c40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.440+0000 7ff955d80640 1 --2- 172.21.15.7:0/1304922607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9501029f0 0x7ff950102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.440+0000 7ff955d80640 1 --2- 172.21.15.7:0/1304922607 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9500ffa20 0x7ff9500ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.440+0000 7ff955d80640 1 -- 172.21.15.7:0/1304922607 >> 172.21.15.7:0/1304922607 conn(0x7ff9500fb350 msgr2=0x7ff9500fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.440+0000 7ff955d80640 1 -- 172.21.15.7:0/1304922607 shutdown_connections 2024-03-23T13:17:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.440+0000 7ff955d80640 1 -- 172.21.15.7:0/1304922607 wait complete. 2024-03-23T13:17:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.441+0000 7ff955d80640 1 Processor -- start 2024-03-23T13:17:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff955d80640 1 -- start start 2024-03-23T13:17:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff955d80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500ffa20 0x7ff950102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff955d80640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9501029f0 0x7ff9501009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff955d80640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9501033b0 0x7ff950100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff955d80640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff950101640 con 0x7ff9500ffa20 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff955d80640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501017b0 con 0x7ff9501029f0 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff955d80640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501a1810 con 0x7ff9501033b0 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500ffa20 0x7ff950102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500ffa20 0x7ff950102360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39746/0 (socket says 172.21.15.7:39746) 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94f7fe640 1 -- 172.21.15.7:0/2938156618 learned_addr learned my addr 172.21.15.7:0/2938156618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94effd640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9501029f0 0x7ff9501009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94ffff640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9501033b0 0x7ff950100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94f7fe640 1 -- 172.21.15.7:0/2938156618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9501033b0 msgr2=0x7ff950100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94f7fe640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9501033b0 0x7ff950100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94f7fe640 1 -- 172.21.15.7:0/2938156618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9501029f0 msgr2=0x7ff9501009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94f7fe640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9501029f0 0x7ff9501009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94f7fe640 1 -- 172.21.15.7:0/2938156618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff948009900 con 0x7ff9500ffa20 2024-03-23T13:17:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94ffff640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9501033b0 0x7ff950100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:17:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.442+0000 7ff94f7fe640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500ffa20 0x7ff950102360 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff94000d980 tx=0x7ff94000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.443+0000 7ff94cff9640 1 -- 172.21.15.7:0/2938156618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940009c30 con 0x7ff9500ffa20 2024-03-23T13:17:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.443+0000 7ff955d80640 1 -- 172.21.15.7:0/2938156618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9501a1ac0 con 0x7ff9500ffa20 2024-03-23T13:17:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.443+0000 7ff94cff9640 1 -- 172.21.15.7:0/2938156618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff940009d90 con 0x7ff9500ffa20 2024-03-23T13:17:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.443+0000 7ff94cff9640 1 -- 172.21.15.7:0/2938156618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94000f790 con 0x7ff9500ffa20 2024-03-23T13:17:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.443+0000 7ff955d80640 1 -- 172.21.15.7:0/2938156618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9501a1fd0 con 0x7ff9500ffa20 2024-03-23T13:17:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.444+0000 7ff955d80640 1 -- 172.21.15.7:0/2938156618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff950108b20 con 0x7ff9500ffa20 2024-03-23T13:17:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.445+0000 7ff94cff9640 1 -- 172.21.15.7:0/2938156618 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7ff940010ad0 con 0x7ff9500ffa20 2024-03-23T13:17:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.446+0000 7ff94cff9640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9380a20c0 0x7ff9380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.446+0000 7ff94effd640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9380a20c0 0x7ff9380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.446+0000 7ff94cff9640 1 -- 172.21.15.7:0/2938156618 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff9400cd810 con 0x7ff9500ffa20 2024-03-23T13:17:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.446+0000 7ff94effd640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9380a20c0 0x7ff9380a4580 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9501020d0 tx=0x7ff944005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:13.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.453+0000 7ff94cff9640 1 -- 172.21.15.7:0/2938156618 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff940096c80 con 0x7ff9500ffa20 2024-03-23T13:17:13.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.687+0000 7ff955d80640 1 -- 172.21.15.7:0/2938156618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff95010d170 con 0x7ff9500ffa20 2024-03-23T13:17:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.688+0000 7ff94cff9640 1 -- 172.21.15.7:0/2938156618 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7ff94009a340 con 0x7ff9500ffa20 2024-03-23T13:17:13.687 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:13.687 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T13:17:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 4 2024-03-23T13:17:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 -- 172.21.15.7:0/2938156618 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9380a20c0 msgr2=0x7ff9380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9380a20c0 0x7ff9380a4580 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9501020d0 tx=0x7ff944005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 -- 172.21.15.7:0/2938156618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500ffa20 msgr2=0x7ff950102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500ffa20 0x7ff950102360 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff94000d980 tx=0x7ff94000de40 comp rx=0 tx=0).stop 2024-03-23T13:17:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 -- 172.21.15.7:0/2938156618 shutdown_connections 2024-03-23T13:17:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff9380a20c0 0x7ff9380a4580 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9501033b0 0x7ff950100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9501029f0 0x7ff9501009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 --2- 172.21.15.7:0/2938156618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500ffa20 0x7ff950102360 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 -- 172.21.15.7:0/2938156618 >> 172.21.15.7:0/2938156618 conn(0x7ff9500fb350 msgr2=0x7ff950105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 -- 172.21.15.7:0/2938156618 shutdown_connections 2024-03-23T13:17:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:13.690+0000 7ff955d80640 1 -- 172.21.15.7:0/2938156618 wait complete. 2024-03-23T13:17:13.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:13 smithi155 ceph-mon[36157]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:13.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:13 smithi007 ceph-mon[32184]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:13.826 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:17:13 smithi007 systemd[1]: Starting Ceph mds.d for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:17:13.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:13 smithi130 ceph-mon[36225]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:14.313 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:17:13 smithi007 podman[72786]: 2024-03-23 13:17:13.883005238 +0000 UTC m=+0.018637323 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:17:14.313 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:17:13 smithi007 podman[72786]: 2024-03-23 13:17:13.995013852 +0000 UTC m=+0.130645968 container create d1feeb764dee2c0c5f32ae9182c6e7f672f8fc68f8acd2652b64ea404386baf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:17:14.313 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:17:14 smithi007 podman[72786]: 2024-03-23 13:17:14.314639262 +0000 UTC m=+0.450271349 container init d1feeb764dee2c0c5f32ae9182c6e7f672f8fc68f8acd2652b64ea404386baf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-d, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-23T13:17:14.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:14 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2938156618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:14.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:14.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:14.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:14.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:14.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:14.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:14 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:14.577 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:17:14 smithi007 podman[72786]: 2024-03-23 13:17:14.317480053 +0000 UTC m=+0.453112136 container start d1feeb764dee2c0c5f32ae9182c6e7f672f8fc68f8acd2652b64ea404386baf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:17:14.577 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:17:14 smithi007 ceph-mds[72805]: -- 172.21.15.7:0/1578015998 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561458e3c1a0 con 0x561458e12c00 2024-03-23T13:17:14.577 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:17:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-d[72801]: starting mds.d at 2024-03-23T13:17:14.577 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:17:14 smithi007 bash[72786]: d1feeb764dee2c0c5f32ae9182c6e7f672f8fc68f8acd2652b64ea404386baf1 2024-03-23T13:17:14.577 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:17:14 smithi007 systemd[1]: Started Ceph mds.d for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:14.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:14 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2938156618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:14.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:14.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:14.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:14.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:14.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:14.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:14 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:14.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:14 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2938156618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:14.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:14.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:14.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:14.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:14.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:14.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:14 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:15.693 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:15 smithi130 ceph-mon[36225]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:15.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:15 smithi130 ceph-mon[36225]: Deploying daemon mds.e on smithi130 2024-03-23T13:17:15.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:15 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.7:6836/746475377,v1:172.21.15.7:6837/746475377] up:boot 2024-03-23T13:17:15.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:15 smithi130 ceph-mon[36225]: fsmap 4 up:standby 2024-03-23T13:17:15.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:15 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T13:17:15.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:15.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:15 smithi155 ceph-mon[36157]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:15.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:15 smithi155 ceph-mon[36157]: Deploying daemon mds.e on smithi130 2024-03-23T13:17:15.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:15 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.7:6836/746475377,v1:172.21.15.7:6837/746475377] up:boot 2024-03-23T13:17:15.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:15 smithi155 ceph-mon[36157]: fsmap 4 up:standby 2024-03-23T13:17:15.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:15 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T13:17:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.820+0000 7fabcc7af640 1 -- 172.21.15.7:0/546736620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabc4100650 msgr2=0x7fabc4100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.820+0000 7fabcc7af640 1 --2- 172.21.15.7:0/546736620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabc4100650 0x7fabc4100a70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009900 tx=0x7fabc002bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.820+0000 7fabcc7af640 1 -- 172.21.15.7:0/546736620 shutdown_connections 2024-03-23T13:17:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.820+0000 7fabcc7af640 1 --2- 172.21.15.7:0/546736620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabc4102200 0x7fabc4108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.820+0000 7fabcc7af640 1 --2- 172.21.15.7:0/546736620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabc4101840 0x7fabc4101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.820+0000 7fabcc7af640 1 --2- 172.21.15.7:0/546736620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabc4100650 0x7fabc4100a70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.820+0000 7fabcc7af640 1 -- 172.21.15.7:0/546736620 >> 172.21.15.7:0/546736620 conn(0x7fabc40fbbc0 msgr2=0x7fabc40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.820+0000 7fabcc7af640 1 -- 172.21.15.7:0/546736620 shutdown_connections 2024-03-23T13:17:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.820+0000 7fabcc7af640 1 -- 172.21.15.7:0/546736620 wait complete. 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabcc7af640 1 Processor -- start 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabcc7af640 1 -- start start 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabcc7af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabc4100650 0x7fabc4197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabcc7af640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabc4101840 0x7fabc4197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabcc7af640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabc4102200 0x7fabc419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabcc7af640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fabc419f330 con 0x7fabc4100650 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabcc7af640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fabc419f4a0 con 0x7fabc4101840 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabcc7af640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fabc419f660 con 0x7fabc4102200 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabca54b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabc4100650 0x7fabc4197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabca54b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabc4100650 0x7fabc4197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53892/0 (socket says 172.21.15.7:53892) 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabca54b640 1 -- 172.21.15.7:0/713928668 learned_addr learned my addr 172.21.15.7:0/713928668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabcad4c640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabc4102200 0x7fabc419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabc9d4a640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabc4101840 0x7fabc4197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabca54b640 1 -- 172.21.15.7:0/713928668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabc4102200 msgr2=0x7fabc419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabca54b640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabc4102200 0x7fabc419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabca54b640 1 -- 172.21.15.7:0/713928668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabc4101840 msgr2=0x7fabc4197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabca54b640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabc4101840 0x7fabc4197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabca54b640 1 -- 172.21.15.7:0/713928668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc0009560 con 0x7fabc4100650 2024-03-23T13:17:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.821+0000 7fabca54b640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabc4100650 0x7fabc4197840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fabc0005ab0 tx=0x7fabc0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.822+0000 7fabbb7fe640 1 -- 172.21.15.7:0/713928668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0004aa0 con 0x7fabc4100650 2024-03-23T13:17:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.822+0000 7fabbb7fe640 1 -- 172.21.15.7:0/713928668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc00052d0 con 0x7fabc4100650 2024-03-23T13:17:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.822+0000 7fabcc7af640 1 -- 172.21.15.7:0/713928668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc419f8e0 con 0x7fabc4100650 2024-03-23T13:17:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.822+0000 7fabbb7fe640 1 -- 172.21.15.7:0/713928668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0003e80 con 0x7fabc4100650 2024-03-23T13:17:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.822+0000 7fabcc7af640 1 -- 172.21.15.7:0/713928668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc419fdf0 con 0x7fabc4100650 2024-03-23T13:17:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.823+0000 7fabcc7af640 1 -- 172.21.15.7:0/713928668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc410c030 con 0x7fabc4100650 2024-03-23T13:17:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.824+0000 7fabbb7fe640 1 -- 172.21.15.7:0/713928668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fabc0005440 con 0x7fabc4100650 2024-03-23T13:17:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.824+0000 7fabbb7fe640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fabb00a20c0 0x7fabb00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.825+0000 7fabc9d4a640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fabb00a20c0 0x7fabb00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.825+0000 7fabbb7fe640 1 -- 172.21.15.7:0/713928668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fabc00efd00 con 0x7fabc4100650 2024-03-23T13:17:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.825+0000 7fabc9d4a640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fabb00a20c0 0x7fabb00a4580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabc4198d50 tx=0x7fabb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:15.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:15 smithi007 ceph-mon[32184]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:15.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:15 smithi007 ceph-mon[32184]: Deploying daemon mds.e on smithi130 2024-03-23T13:17:15.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:15 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.7:6836/746475377,v1:172.21.15.7:6837/746475377] up:boot 2024-03-23T13:17:15.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:15 smithi007 ceph-mon[32184]: fsmap 4 up:standby 2024-03-23T13:17:15.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:15 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T13:17:15.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:15.829+0000 7fabbb7fe640 1 -- 172.21.15.7:0/713928668 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fabc00b9170 con 0x7fabc4100650 2024-03-23T13:17:16.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.052+0000 7fabcc7af640 1 -- 172.21.15.7:0/713928668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabc410c1f0 con 0x7fabc4100650 2024-03-23T13:17:16.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.053+0000 7fabbb7fe640 1 -- 172.21.15.7:0/713928668 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7fabc00bc830 con 0x7fabc4100650 2024-03-23T13:17:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:16.052 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T13:17:16.052 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 5 2024-03-23T13:17:16.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 -- 172.21.15.7:0/713928668 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fabb00a20c0 msgr2=0x7fabb00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:16.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fabb00a20c0 0x7fabb00a4580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabc4198d50 tx=0x7fabb4005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:16.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 -- 172.21.15.7:0/713928668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabc4100650 msgr2=0x7fabc4197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:16.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabc4100650 0x7fabc4197840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fabc0005ab0 tx=0x7fabc0004270 comp rx=0 tx=0).stop 2024-03-23T13:17:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 -- 172.21.15.7:0/713928668 shutdown_connections 2024-03-23T13:17:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fabb00a20c0 0x7fabb00a4580 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabc4102200 0x7fabc419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabc4101840 0x7fabc4197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 --2- 172.21.15.7:0/713928668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabc4100650 0x7fabc4197840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 -- 172.21.15.7:0/713928668 >> 172.21.15.7:0/713928668 conn(0x7fabc40fbbc0 msgr2=0x7fabc4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 -- 172.21.15.7:0/713928668 shutdown_connections 2024-03-23T13:17:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:16.055+0000 7fabcc7af640 1 -- 172.21.15.7:0/713928668 wait complete. 2024-03-23T13:17:16.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:16 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/713928668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:16.695 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:17:16 smithi130 systemd[1]: Starting Ceph mds.e for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:17:16.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:16 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/713928668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:16.963 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:16 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/713928668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:17.075 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:17:16 smithi130 podman[66390]: 2024-03-23 13:17:16.799379203 +0000 UTC m=+0.096122328 container create c0da684a212996c9e2fae0ce5a73a08f0bddb0cb0859815982e5cfe5ad54e6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:17:17.075 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:17:16 smithi130 podman[66390]: 2024-03-23 13:17:16.72106634 +0000 UTC m=+0.017809487 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:17:17.075 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:17:17 smithi130 podman[66390]: 2024-03-23 13:17:17.077003456 +0000 UTC m=+0.373755213 container init c0da684a212996c9e2fae0ce5a73a08f0bddb0cb0859815982e5cfe5ad54e6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-23T13:17:17.444 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:17:17 smithi130 podman[66390]: 2024-03-23 13:17:17.07974374 +0000 UTC m=+0.376486884 container start c0da684a212996c9e2fae0ce5a73a08f0bddb0cb0859815982e5cfe5ad54e6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-23T13:17:17.444 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:17:17 smithi130 ceph-mds[66409]: -- 172.21.15.130:0/4165776557 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d39ee581a0 con 0x55d39ee2ec00 2024-03-23T13:17:17.444 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:17:17 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e[66405]: starting mds.e at 2024-03-23T13:17:17.444 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:17:17 smithi130 bash[66390]: c0da684a212996c9e2fae0ce5a73a08f0bddb0cb0859815982e5cfe5ad54e6fb 2024-03-23T13:17:17.444 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:17:17 smithi130 systemd[1]: Started Ceph mds.e for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:17.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:17 smithi007 ceph-mon[32184]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:17.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:17.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:17.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:17.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:17.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:17.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:17 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:17.904 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:17 smithi155 ceph-mon[36157]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:17.904 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:17.904 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:17.905 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:17.905 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:17.905 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:17.905 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:17 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:17.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:17 smithi130 ceph-mon[36225]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:17.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:17.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:17.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:17.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:17.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:17.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:17 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:18.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.182+0000 7f1cf0855640 1 -- 172.21.15.7:0/3736085888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec102260 msgr2=0x7f1cec108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.182+0000 7f1cf0855640 1 --2- 172.21.15.7:0/3736085888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec102260 0x7f1cec108af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009ca0 tx=0x7f1ce402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.182+0000 7f1cf0855640 1 -- 172.21.15.7:0/3736085888 shutdown_connections 2024-03-23T13:17:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.182+0000 7f1cf0855640 1 --2- 172.21.15.7:0/3736085888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec102260 0x7f1cec108af0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.182+0000 7f1cf0855640 1 --2- 172.21.15.7:0/3736085888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cec1018a0 0x7f1cec101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.182+0000 7f1cf0855640 1 --2- 172.21.15.7:0/3736085888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cec1006b0 0x7f1cec100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.182+0000 7f1cf0855640 1 -- 172.21.15.7:0/3736085888 >> 172.21.15.7:0/3736085888 conn(0x7f1cec0fbc40 msgr2=0x7f1cec0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.182+0000 7f1cf0855640 1 -- 172.21.15.7:0/3736085888 shutdown_connections 2024-03-23T13:17:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.182+0000 7f1cf0855640 1 -- 172.21.15.7:0/3736085888 wait complete. 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.183+0000 7f1cf0855640 1 Processor -- start 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.183+0000 7f1cf0855640 1 -- start start 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.183+0000 7f1cf0855640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cec1006b0 0x7f1cec197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.183+0000 7f1cf0855640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cec1018a0 0x7f1cec197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.183+0000 7f1cf0855640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec102260 0x7f1cec19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.183+0000 7f1cf0855640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec19f250 con 0x7f1cec102260 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.183+0000 7f1cf0855640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec19f3c0 con 0x7f1cec1018a0 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.183+0000 7f1cf0855640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec19f580 con 0x7f1cec1006b0 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.183+0000 7f1cead9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec102260 0x7f1cec19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cea59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cec1006b0 0x7f1cec197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cea59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cec1006b0 0x7f1cec197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:47198/0 (socket says 172.21.15.7:47198) 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cead9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec102260 0x7f1cec19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53912/0 (socket says 172.21.15.7:53912) 2024-03-23T13:17:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cea59c640 1 -- 172.21.15.7:0/178870266 learned_addr learned my addr 172.21.15.7:0/178870266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1ce1d9b640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cec1018a0 0x7f1cec197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cead9d640 1 -- 172.21.15.7:0/178870266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cec1006b0 msgr2=0x7f1cec197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cead9d640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cec1006b0 0x7f1cec197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cead9d640 1 -- 172.21.15.7:0/178870266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cec1018a0 msgr2=0x7f1cec197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cead9d640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cec1018a0 0x7f1cec197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cead9d640 1 -- 172.21.15.7:0/178870266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce4009900 con 0x7f1cec102260 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1ce1d9b640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cec1018a0 0x7f1cec197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cea59c640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cec1006b0 0x7f1cec197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cead9d640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec102260 0x7f1cec19ed10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ce402c630 tx=0x7f1ce4034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1ce3fff640 1 -- 172.21.15.7:0/178870266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4032eb0 con 0x7f1cec102260 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1ce3fff640 1 -- 172.21.15.7:0/178870266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce4003e50 con 0x7f1cec102260 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cf0855640 1 -- 172.21.15.7:0/178870266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec19f800 con 0x7f1cec102260 2024-03-23T13:17:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1ce3fff640 1 -- 172.21.15.7:0/178870266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4040450 con 0x7f1cec102260 2024-03-23T13:17:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.184+0000 7f1cf0855640 1 -- 172.21.15.7:0/178870266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec19fc80 con 0x7f1cec102260 2024-03-23T13:17:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.185+0000 7f1cf0855640 1 -- 172.21.15.7:0/178870266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec10c050 con 0x7f1cec102260 2024-03-23T13:17:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.187+0000 7f1ce3fff640 1 -- 172.21.15.7:0/178870266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f1ce400a040 con 0x7f1cec102260 2024-03-23T13:17:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.187+0000 7f1ce3fff640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1cd80a20c0 0x7f1cd80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.187+0000 7f1cea59c640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1cd80a20c0 0x7f1cd80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.187+0000 7f1ce3fff640 1 -- 172.21.15.7:0/178870266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1ce40f0630 con 0x7f1cec102260 2024-03-23T13:17:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.188+0000 7f1cea59c640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1cd80a20c0 0x7f1cd80a4580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc011550 tx=0x7f1cdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:18.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.192+0000 7f1ce3fff640 1 -- 172.21.15.7:0/178870266 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1ce40b9aa0 con 0x7f1cec102260 2024-03-23T13:17:18.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.411+0000 7f1cf0855640 1 -- 172.21.15.7:0/178870266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cec1a0120 con 0x7f1cec102260 2024-03-23T13:17:18.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.412+0000 7f1ce3fff640 1 -- 172.21.15.7:0/178870266 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f1ce40bd160 con 0x7f1cec102260 2024-03-23T13:17:18.411 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-23T13:17:18.411 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 6 2024-03-23T13:17:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.413+0000 7f1cf0855640 1 -- 172.21.15.7:0/178870266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1cd80a20c0 msgr2=0x7f1cd80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.413+0000 7f1cf0855640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1cd80a20c0 0x7f1cd80a4580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc011550 tx=0x7f1cdc005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.413+0000 7f1cf0855640 1 -- 172.21.15.7:0/178870266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec102260 msgr2=0x7f1cec19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.413+0000 7f1cf0855640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec102260 0x7f1cec19ed10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ce402c630 tx=0x7f1ce4034d40 comp rx=0 tx=0).stop 2024-03-23T13:17:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.414+0000 7f1cf0855640 1 -- 172.21.15.7:0/178870266 shutdown_connections 2024-03-23T13:17:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.414+0000 7f1cf0855640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1cd80a20c0 0x7f1cd80a4580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.414+0000 7f1cf0855640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec102260 0x7f1cec19ed10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.414+0000 7f1cf0855640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cec1018a0 0x7f1cec197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.414+0000 7f1cf0855640 1 --2- 172.21.15.7:0/178870266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cec1006b0 0x7f1cec197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.414+0000 7f1cf0855640 1 -- 172.21.15.7:0/178870266 >> 172.21.15.7:0/178870266 conn(0x7f1cec0fbc40 msgr2=0x7f1cec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.414+0000 7f1cf0855640 1 -- 172.21.15.7:0/178870266 shutdown_connections 2024-03-23T13:17:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:18.414+0000 7f1cf0855640 1 -- 172.21.15.7:0/178870266 wait complete. 2024-03-23T13:17:18.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:18 smithi007 ceph-mon[32184]: Deploying daemon mds.f on smithi155 2024-03-23T13:17:18.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:18 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.130:6834/2599360099,v1:172.21.15.130:6835/2599360099] up:boot 2024-03-23T13:17:18.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:18 smithi007 ceph-mon[32184]: fsmap 5 up:standby 2024-03-23T13:17:18.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:18 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T13:17:18.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:18 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/178870266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:18.925 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:18 smithi155 ceph-mon[36157]: Deploying daemon mds.f on smithi155 2024-03-23T13:17:18.925 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:18 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.130:6834/2599360099,v1:172.21.15.130:6835/2599360099] up:boot 2024-03-23T13:17:18.925 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:18 smithi155 ceph-mon[36157]: fsmap 5 up:standby 2024-03-23T13:17:18.925 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:18 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T13:17:18.926 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:18 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/178870266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:18.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:18 smithi130 ceph-mon[36225]: Deploying daemon mds.f on smithi155 2024-03-23T13:17:18.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:18 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.130:6834/2599360099,v1:172.21.15.130:6835/2599360099] up:boot 2024-03-23T13:17:18.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:18 smithi130 ceph-mon[36225]: fsmap 5 up:standby 2024-03-23T13:17:18.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:18 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T13:17:18.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:18 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/178870266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:19.537 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:17:19 smithi155 systemd[1]: Starting Ceph mds.f for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:17:19.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:19 smithi007 ceph-mon[32184]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:19.888 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:19 smithi155 ceph-mon[36157]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:19.888 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:17:19 smithi155 podman[66524]: 2024-03-23 13:17:19.508357138 +0000 UTC m=+0.017655437 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:17:19.889 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:17:19 smithi155 podman[66524]: 2024-03-23 13:17:19.629565669 +0000 UTC m=+0.138863982 container create 952c73e93ab4205643e2545c73e441fb31b8c07bbb3bd5989c26edf24553b439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319) 2024-03-23T13:17:19.943 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:19 smithi130 ceph-mon[36225]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:20.287 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:17:19 smithi155 podman[66524]: 2024-03-23 13:17:19.890374298 +0000 UTC m=+0.399672592 container init 952c73e93ab4205643e2545c73e441fb31b8c07bbb3bd5989c26edf24553b439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:17:20.287 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:17:19 smithi155 podman[66524]: 2024-03-23 13:17:19.892950926 +0000 UTC m=+0.402249220 container start 952c73e93ab4205643e2545c73e441fb31b8c07bbb3bd5989c26edf24553b439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:17:20.287 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:17:19 smithi155 ceph-mds[66544]: -- 172.21.15.155:0/1863101896 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee0d8961a0 con 0x55ee0d86cc00 2024-03-23T13:17:20.288 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:17:19 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f[66540]: starting mds.f at 2024-03-23T13:17:20.288 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:17:19 smithi155 bash[66524]: 952c73e93ab4205643e2545c73e441fb31b8c07bbb3bd5989c26edf24553b439 2024-03-23T13:17:20.288 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:17:19 smithi155 systemd[1]: Started Ceph mds.f for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:20.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.541+0000 7feda7f93640 1 -- 172.21.15.7:0/2339019367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda00fffc0 msgr2=0x7feda01003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.541+0000 7feda7f93640 1 --2- 172.21.15.7:0/2339019367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda00fffc0 0x7feda01003e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009900 tx=0x7fed9c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.541+0000 7feda7f93640 1 -- 172.21.15.7:0/2339019367 shutdown_connections 2024-03-23T13:17:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.541+0000 7feda7f93640 1 --2- 172.21.15.7:0/2339019367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feda01012c0 0x7feda0101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.541+0000 7feda7f93640 1 --2- 172.21.15.7:0/2339019367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feda0100920 0x7feda0100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.541+0000 7feda7f93640 1 --2- 172.21.15.7:0/2339019367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda00fffc0 0x7feda01003e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.541+0000 7feda7f93640 1 -- 172.21.15.7:0/2339019367 >> 172.21.15.7:0/2339019367 conn(0x7feda00fbbc0 msgr2=0x7feda00fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.541+0000 7feda7f93640 1 -- 172.21.15.7:0/2339019367 shutdown_connections 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.541+0000 7feda7f93640 1 -- 172.21.15.7:0/2339019367 wait complete. 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda7f93640 1 Processor -- start 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda7f93640 1 -- start start 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda7f93640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feda00fffc0 0x7feda0199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda7f93640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feda0100920 0x7feda0199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda7f93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda01012c0 0x7feda01a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda7f93640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feda01a1410 con 0x7feda01012c0 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda7f93640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7feda01a1580 con 0x7feda0100920 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda7f93640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feda01a1740 con 0x7feda00fffc0 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda6530640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda01012c0 0x7feda01a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda6530640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda01012c0 0x7feda01a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53936/0 (socket says 172.21.15.7:53936) 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda6530640 1 -- 172.21.15.7:0/286088662 learned_addr learned my addr 172.21.15.7:0/286088662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda5d2f640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feda00fffc0 0x7feda0199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.542+0000 7feda552e640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feda0100920 0x7feda0199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7feda6530640 1 -- 172.21.15.7:0/286088662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feda00fffc0 msgr2=0x7feda0199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7feda6530640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feda00fffc0 0x7feda0199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7feda6530640 1 -- 172.21.15.7:0/286088662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feda0100920 msgr2=0x7feda0199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7feda6530640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feda0100920 0x7feda0199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7feda6530640 1 -- 172.21.15.7:0/286088662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c009560 con 0x7feda01012c0 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7feda552e640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feda0100920 0x7feda0199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7feda6530640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda01012c0 0x7feda01a0ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fed9800eaa0 tx=0x7fed9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7fed96ffd640 1 -- 172.21.15.7:0/286088662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9800cdb0 con 0x7feda01012c0 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7fed96ffd640 1 -- 172.21.15.7:0/286088662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed98005020 con 0x7feda01012c0 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7feda7f93640 1 -- 172.21.15.7:0/286088662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda01a1a20 con 0x7feda01012c0 2024-03-23T13:17:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7fed96ffd640 1 -- 172.21.15.7:0/286088662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed980116b0 con 0x7feda01012c0 2024-03-23T13:17:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.543+0000 7feda7f93640 1 -- 172.21.15.7:0/286088662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda01a1f30 con 0x7feda01012c0 2024-03-23T13:17:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.544+0000 7feda7f93640 1 -- 172.21.15.7:0/286088662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda010ae20 con 0x7feda01012c0 2024-03-23T13:17:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.545+0000 7fed96ffd640 1 -- 172.21.15.7:0/286088662 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fed98005520 con 0x7feda01012c0 2024-03-23T13:17:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.546+0000 7fed96ffd640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed8c0a20c0 0x7fed8c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.546+0000 7feda5d2f640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed8c0a20c0 0x7fed8c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.546+0000 7fed96ffd640 1 -- 172.21.15.7:0/286088662 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fed980cdaf0 con 0x7feda01012c0 2024-03-23T13:17:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.546+0000 7feda5d2f640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed8c0a20c0 0x7fed8c0a4580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009230 tx=0x7fed9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.552+0000 7fed96ffd640 1 -- 172.21.15.7:0/286088662 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fed98096ec0 con 0x7feda01012c0 2024-03-23T13:17:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.772+0000 7feda7f93640 1 -- 172.21.15.7:0/286088662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feda010f330 con 0x7feda01012c0 2024-03-23T13:17:20.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.773+0000 7fed96ffd640 1 -- 172.21.15.7:0/286088662 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7fed9809a580 con 0x7feda01012c0 2024-03-23T13:17:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:20.772 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-23T13:17:20.772 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2024-03-23T13:17:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 -- 172.21.15.7:0/286088662 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed8c0a20c0 msgr2=0x7fed8c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed8c0a20c0 0x7fed8c0a4580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009230 tx=0x7fed9c005fd0 comp rx=0 tx=0).stop 2024-03-23T13:17:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 -- 172.21.15.7:0/286088662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda01012c0 msgr2=0x7feda01a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda01012c0 0x7feda01a0ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fed9800eaa0 tx=0x7fed9800ef60 comp rx=0 tx=0).stop 2024-03-23T13:17:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 -- 172.21.15.7:0/286088662 shutdown_connections 2024-03-23T13:17:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fed8c0a20c0 0x7fed8c0a4580 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda01012c0 0x7feda01a0ed0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:20.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feda0100920 0x7feda0199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:20.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 --2- 172.21.15.7:0/286088662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feda00fffc0 0x7feda0199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:20.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 -- 172.21.15.7:0/286088662 >> 172.21.15.7:0/286088662 conn(0x7feda00fbbc0 msgr2=0x7feda00fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:20.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 -- 172.21.15.7:0/286088662 shutdown_connections 2024-03-23T13:17:20.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:20.775+0000 7feda7f93640 1 -- 172.21.15.7:0/286088662 wait complete. 2024-03-23T13:17:21.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:21.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: Deploying daemon mds.c on smithi155 2024-03-23T13:17:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] up:boot 2024-03-23T13:17:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: fsmap 6 up:standby 2024-03-23T13:17:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T13:17:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:21 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/286088662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:21.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:21.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:21.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:21.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:21.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:21.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:21.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:21.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: Deploying daemon mds.c on smithi155 2024-03-23T13:17:21.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] up:boot 2024-03-23T13:17:21.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: fsmap 6 up:standby 2024-03-23T13:17:21.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T13:17:21.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:21 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/286088662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:21.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:21.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:21.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:21.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:21.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:21.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:21.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:21.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: Deploying daemon mds.c on smithi155 2024-03-23T13:17:21.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] up:boot 2024-03-23T13:17:21.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: fsmap 6 up:standby 2024-03-23T13:17:21.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T13:17:21.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:21 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/286088662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:22.288 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:17:22 smithi155 systemd[1]: Starting Ceph mds.c for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:17:22.735 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:17:22 smithi155 podman[66835]: 2024-03-23 13:17:22.319489592 +0000 UTC m=+0.017583870 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:17:22.735 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:17:22 smithi155 podman[66835]: 2024-03-23 13:17:22.508861325 +0000 UTC m=+0.206955604 container create 17af4a18e4c41fc31d68f58bb824873d913e542516da0df6ee58e196dd4f3a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-23T13:17:22.735 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:17:22 smithi155 podman[66835]: 2024-03-23 13:17:22.736433623 +0000 UTC m=+0.434527892 container init 17af4a18e4c41fc31d68f58bb824873d913e542516da0df6ee58e196dd4f3a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:17:22.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.913+0000 7f3649ff4640 1 -- 172.21.15.7:0/1600702768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36441018a0 msgr2=0x7f3644101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.913+0000 7f3649ff4640 1 --2- 172.21.15.7:0/1600702768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36441018a0 0x7f3644101d20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3634009900 tx=0x7f363402bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.913+0000 7f3649ff4640 1 -- 172.21.15.7:0/1600702768 shutdown_connections 2024-03-23T13:17:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.913+0000 7f3649ff4640 1 --2- 172.21.15.7:0/1600702768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3644102260 0x7f3644108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.913+0000 7f3649ff4640 1 --2- 172.21.15.7:0/1600702768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36441018a0 0x7f3644101d20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.913+0000 7f3649ff4640 1 --2- 172.21.15.7:0/1600702768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36441006b0 0x7f3644100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.913+0000 7f3649ff4640 1 -- 172.21.15.7:0/1600702768 >> 172.21.15.7:0/1600702768 conn(0x7f36440fbc40 msgr2=0x7f36440fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.913+0000 7f3649ff4640 1 -- 172.21.15.7:0/1600702768 shutdown_connections 2024-03-23T13:17:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.913+0000 7f3649ff4640 1 -- 172.21.15.7:0/1600702768 wait complete. 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3649ff4640 1 Processor -- start 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3649ff4640 1 -- start start 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3649ff4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36441006b0 0x7f364406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3649ff4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36441018a0 0x7f364419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3649ff4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3644102260 0x7f364419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3649ff4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f364419f270 con 0x7f3644102260 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3649ff4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f364419f3e0 con 0x7f36441006b0 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3649ff4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f364419f5a0 con 0x7f36441018a0 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3643fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3644102260 0x7f364419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3643fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3644102260 0x7f364419ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53958/0 (socket says 172.21.15.7:53958) 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3643fff640 1 -- 172.21.15.7:0/3191816878 learned_addr learned my addr 172.21.15.7:0/3191816878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f36437fe640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36441006b0 0x7f364406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.914+0000 7f3642ffd640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36441018a0 0x7f364419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3643fff640 1 -- 172.21.15.7:0/3191816878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36441018a0 msgr2=0x7f364419c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3643fff640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36441018a0 0x7f364419c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3643fff640 1 -- 172.21.15.7:0/3191816878 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36441006b0 msgr2=0x7f364406a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3643fff640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36441006b0 0x7f364406a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3643fff640 1 -- 172.21.15.7:0/3191816878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3634009560 con 0x7f3644102260 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3643fff640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3644102260 0x7f364419ea90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f363c009fd0 tx=0x7f363c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3640ff9640 1 -- 172.21.15.7:0/3191816878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c00bea0 con 0x7f3644102260 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3640ff9640 1 -- 172.21.15.7:0/3191816878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363c009940 con 0x7f3644102260 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3640ff9640 1 -- 172.21.15.7:0/3191816878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c0106c0 con 0x7f3644102260 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3649ff4640 1 -- 172.21.15.7:0/3191816878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364419f820 con 0x7f3644102260 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.915+0000 7f3649ff4640 1 -- 172.21.15.7:0/3191816878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364419fcb0 con 0x7f3644102260 2024-03-23T13:17:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.916+0000 7f3649ff4640 1 -- 172.21.15.7:0/3191816878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3644107970 con 0x7f3644102260 2024-03-23T13:17:22.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.917+0000 7f3640ff9640 1 -- 172.21.15.7:0/3191816878 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f363c004240 con 0x7f3644102260 2024-03-23T13:17:22.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.918+0000 7f3640ff9640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f362c0a20c0 0x7f362c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:22.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.918+0000 7f36437fe640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f362c0a20c0 0x7f362c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:22.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.918+0000 7f3640ff9640 1 -- 172.21.15.7:0/3191816878 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f363c0cd470 con 0x7f3644102260 2024-03-23T13:17:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.918+0000 7f36437fe640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f362c0a20c0 0x7f362c0a4580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3644101700 tx=0x7f3638005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:22.923+0000 7f3640ff9640 1 -- 172.21.15.7:0/3191816878 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f363c096990 con 0x7f3644102260 2024-03-23T13:17:23.037 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:17:22 smithi155 podman[66835]: 2024-03-23 13:17:22.739345836 +0000 UTC m=+0.437440104 container start 17af4a18e4c41fc31d68f58bb824873d913e542516da0df6ee58e196dd4f3a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-23T13:17:23.037 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:17:22 smithi155 ceph-mds[66854]: -- 172.21.15.155:0/3130385211 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f14daca1a0 con 0x55f14daa0c00 2024-03-23T13:17:23.037 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:17:22 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c[66850]: starting mds.c at 2024-03-23T13:17:23.037 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:17:22 smithi155 bash[66835]: 17af4a18e4c41fc31d68f58bb824873d913e542516da0df6ee58e196dd4f3a63 2024-03-23T13:17:23.037 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:17:22 smithi155 systemd[1]: Started Ceph mds.c for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.154+0000 7f3649ff4640 1 -- 172.21.15.7:0/3191816878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f364410c2a0 con 0x7f3644102260 2024-03-23T13:17:23.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.155+0000 7f3640ff9640 1 -- 172.21.15.7:0/3191816878 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f363c09a050 con 0x7f3644102260 2024-03-23T13:17:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:23.155 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-23T13:17:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.156+0000 7f3649ff4640 1 -- 172.21.15.7:0/3191816878 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f362c0a20c0 msgr2=0x7f362c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.156+0000 7f3649ff4640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f362c0a20c0 0x7f362c0a4580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3644101700 tx=0x7f3638005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.156+0000 7f3649ff4640 1 -- 172.21.15.7:0/3191816878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3644102260 msgr2=0x7f364419ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.156+0000 7f3649ff4640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3644102260 0x7f364419ea90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f363c009fd0 tx=0x7f363c00b260 comp rx=0 tx=0).stop 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.156+0000 7f3649ff4640 1 -- 172.21.15.7:0/3191816878 shutdown_connections 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.156+0000 7f3649ff4640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f362c0a20c0 0x7f362c0a4580 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.156+0000 7f3649ff4640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3644102260 0x7f364419ea90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.157+0000 7f3649ff4640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36441018a0 0x7f364419c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.157+0000 7f3649ff4640 1 --2- 172.21.15.7:0/3191816878 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36441006b0 0x7f364406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.157+0000 7f3649ff4640 1 -- 172.21.15.7:0/3191816878 >> 172.21.15.7:0/3191816878 conn(0x7f36440fbc40 msgr2=0x7f3644104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.157+0000 7f3649ff4640 1 -- 172.21.15.7:0/3191816878 shutdown_connections 2024-03-23T13:17:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:23.157+0000 7f3649ff4640 1 -- 172.21.15.7:0/3191816878 wait complete. 2024-03-23T13:17:24.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:24.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:24.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:24.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:24.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:24.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:24.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:24.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] up:boot 2024-03-23T13:17:24.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: fsmap 7 up:standby 2024-03-23T13:17:24.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T13:17:24.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:23 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3191816878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:24.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:24.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:24.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:24.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:24.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:24.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:24.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:24.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] up:boot 2024-03-23T13:17:24.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: fsmap 7 up:standby 2024-03-23T13:17:24.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T13:17:24.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:23 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3191816878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:24.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:24.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:24.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:24.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:24.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:24.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:24.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:24.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] up:boot 2024-03-23T13:17:24.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: fsmap 7 up:standby 2024-03-23T13:17:24.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T13:17:24.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:23 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3191816878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:25.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:25.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:24 smithi130 ceph-mon[36225]: Deploying daemon mds.h on smithi130 2024-03-23T13:17:25.195 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:17:25 smithi130 systemd[1]: Starting Ceph mds.h for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:17:25.198 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:24 smithi007 ceph-mon[32184]: Deploying daemon mds.h on smithi130 2024-03-23T13:17:25.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.285+0000 7fe59cc0e640 1 -- 172.21.15.7:0/710143219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe598100630 msgr2=0x7fe598100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:25.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.285+0000 7fe59cc0e640 1 --2- 172.21.15.7:0/710143219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe598100630 0x7fe598100a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe588009ca0 tx=0x7fe58802c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:25.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.285+0000 7fe59cc0e640 1 -- 172.21.15.7:0/710143219 shutdown_connections 2024-03-23T13:17:25.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.285+0000 7fe59cc0e640 1 --2- 172.21.15.7:0/710143219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe5981021e0 0x7fe598108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:25.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.285+0000 7fe59cc0e640 1 --2- 172.21.15.7:0/710143219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe598101820 0x7fe598101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:25.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.285+0000 7fe59cc0e640 1 --2- 172.21.15.7:0/710143219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe598100630 0x7fe598100a50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:25.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.285+0000 7fe59cc0e640 1 -- 172.21.15.7:0/710143219 >> 172.21.15.7:0/710143219 conn(0x7fe5980fbba0 msgr2=0x7fe5980fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.285+0000 7fe59cc0e640 1 -- 172.21.15.7:0/710143219 shutdown_connections 2024-03-23T13:17:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.285+0000 7fe59cc0e640 1 -- 172.21.15.7:0/710143219 wait complete. 2024-03-23T13:17:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.285+0000 7fe59cc0e640 1 Processor -- start 2024-03-23T13:17:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59cc0e640 1 -- start start 2024-03-23T13:17:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59cc0e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe598100630 0x7fe598197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59cc0e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe598101820 0x7fe598197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59cc0e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5981021e0 0x7fe59819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59cc0e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe59819f370 con 0x7fe598100630 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59cc0e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe59819f4e0 con 0x7fe598101820 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59cc0e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe59819f6a0 con 0x7fe5981021e0 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe598100630 0x7fe598197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe598100630 0x7fe598197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41574/0 (socket says 172.21.15.7:41574) 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59659c640 1 -- 172.21.15.7:0/1090654953 learned_addr learned my addr 172.21.15.7:0/1090654953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe595d9b640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe598101820 0x7fe598197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe596d9d640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5981021e0 0x7fe59819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59659c640 1 -- 172.21.15.7:0/1090654953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5981021e0 msgr2=0x7fe59819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59659c640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5981021e0 0x7fe59819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59659c640 1 -- 172.21.15.7:0/1090654953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe598101820 msgr2=0x7fe598197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59659c640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe598101820 0x7fe598197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59659c640 1 -- 172.21.15.7:0/1090654953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe588009900 con 0x7fe598100630 2024-03-23T13:17:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe595d9b640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe598101820 0x7fe598197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:17:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.286+0000 7fe59659c640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe598100630 0x7fe598197880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe588037040 tx=0x7fe588034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.287+0000 7fe5877fe640 1 -- 172.21.15.7:0/1090654953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588032eb0 con 0x7fe598100630 2024-03-23T13:17:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.287+0000 7fe5877fe640 1 -- 172.21.15.7:0/1090654953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe588003e50 con 0x7fe598100630 2024-03-23T13:17:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.287+0000 7fe59cc0e640 1 -- 172.21.15.7:0/1090654953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59819f920 con 0x7fe598100630 2024-03-23T13:17:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.287+0000 7fe5877fe640 1 -- 172.21.15.7:0/1090654953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588040470 con 0x7fe598100630 2024-03-23T13:17:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.287+0000 7fe59cc0e640 1 -- 172.21.15.7:0/1090654953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59819fdd0 con 0x7fe598100630 2024-03-23T13:17:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.288+0000 7fe59cc0e640 1 -- 172.21.15.7:0/1090654953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59810c080 con 0x7fe598100630 2024-03-23T13:17:25.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:24 smithi155 ceph-mon[36157]: Deploying daemon mds.h on smithi130 2024-03-23T13:17:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.289+0000 7fe5877fe640 1 -- 172.21.15.7:0/1090654953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fe58800a040 con 0x7fe598100630 2024-03-23T13:17:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.289+0000 7fe5877fe640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5800a20c0 0x7fe5800a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.290+0000 7fe595d9b640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5800a20c0 0x7fe5800a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.290+0000 7fe5877fe640 1 -- 172.21.15.7:0/1090654953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe5880f0760 con 0x7fe598100630 2024-03-23T13:17:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.290+0000 7fe595d9b640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5800a20c0 0x7fe5800a4580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe598198d90 tx=0x7fe58c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.294+0000 7fe5877fe640 1 -- 172.21.15.7:0/1090654953 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fe5880b9bd0 con 0x7fe598100630 2024-03-23T13:17:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.517+0000 7fe59cc0e640 1 -- 172.21.15.7:0/1090654953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe59810c290 con 0x7fe598100630 2024-03-23T13:17:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.518+0000 7fe5877fe640 1 -- 172.21.15.7:0/1090654953 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fe5880bd290 con 0x7fe598100630 2024-03-23T13:17:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:25.518 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-23T13:17:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-03-23T13:17:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 -- 172.21.15.7:0/1090654953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5800a20c0 msgr2=0x7fe5800a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5800a20c0 0x7fe5800a4580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe598198d90 tx=0x7fe58c008040 comp rx=0 tx=0).stop 2024-03-23T13:17:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 -- 172.21.15.7:0/1090654953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe598100630 msgr2=0x7fe598197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe598100630 0x7fe598197880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe588037040 tx=0x7fe588034d40 comp rx=0 tx=0).stop 2024-03-23T13:17:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 -- 172.21.15.7:0/1090654953 shutdown_connections 2024-03-23T13:17:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5800a20c0 0x7fe5800a4580 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5981021e0 0x7fe59819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe598101820 0x7fe598197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 --2- 172.21.15.7:0/1090654953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe598100630 0x7fe598197880 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 -- 172.21.15.7:0/1090654953 >> 172.21.15.7:0/1090654953 conn(0x7fe5980fbba0 msgr2=0x7fe598068870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 -- 172.21.15.7:0/1090654953 shutdown_connections 2024-03-23T13:17:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:25.520+0000 7fe59cc0e640 1 -- 172.21.15.7:0/1090654953 wait complete. 2024-03-23T13:17:25.581 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:17:25 smithi130 podman[66699]: 2024-03-23 13:17:25.209286287 +0000 UTC m=+0.018016594 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:17:25.581 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:17:25 smithi130 podman[66699]: 2024-03-23 13:17:25.321814809 +0000 UTC m=+0.130545100 container create 189dca2e132be202fd169aec61d53072db4a3438cfc77551ba86586d823e672b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:17:25.581 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:17:25 smithi130 podman[66699]: 2024-03-23 13:17:25.583165209 +0000 UTC m=+0.391895513 container init 189dca2e132be202fd169aec61d53072db4a3438cfc77551ba86586d823e672b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-23T13:17:25.884 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:17:25 smithi130 podman[66699]: 2024-03-23 13:17:25.586566707 +0000 UTC m=+0.395297008 container start 189dca2e132be202fd169aec61d53072db4a3438cfc77551ba86586d823e672b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-23T13:17:25.884 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:17:25 smithi130 ceph-mds[66718]: -- 172.21.15.130:0/2138732315 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558cd75ea1a0 con 0x558cd75c1000 2024-03-23T13:17:25.884 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:17:25 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h[66714]: starting mds.h at 2024-03-23T13:17:25.884 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:17:25 smithi130 bash[66699]: 189dca2e132be202fd169aec61d53072db4a3438cfc77551ba86586d823e672b 2024-03-23T13:17:25.885 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:17:25 smithi130 systemd[1]: Started Ceph mds.h for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:26.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:25 smithi130 ceph-mon[36225]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:26.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:25 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1090654953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:26.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:25 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:26.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:25 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:26.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:25 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:26.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:25 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:26.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:25 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:26.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:25 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:26.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:25 smithi155 ceph-mon[36157]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:26.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:25 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1090654953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:26.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:25 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:26.287 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:25 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:26.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:25 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:26.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:25 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:26.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:25 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:26.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:25 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:26.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:25 smithi007 ceph-mon[32184]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:26.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:25 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1090654953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:26.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:25 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:26.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:25 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:26.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:25 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:26.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:25 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:26.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:25 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:26.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:25 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:27.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:26 smithi155 ceph-mon[36157]: Deploying daemon mds.i on smithi155 2024-03-23T13:17:27.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:26 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] up:boot 2024-03-23T13:17:27.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:26 smithi155 ceph-mon[36157]: fsmap 8 up:standby 2024-03-23T13:17:27.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:26 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T13:17:27.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:26 smithi130 ceph-mon[36225]: Deploying daemon mds.i on smithi155 2024-03-23T13:17:27.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:26 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] up:boot 2024-03-23T13:17:27.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:26 smithi130 ceph-mon[36225]: fsmap 8 up:standby 2024-03-23T13:17:27.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:26 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T13:17:27.228 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:26 smithi007 ceph-mon[32184]: Deploying daemon mds.i on smithi155 2024-03-23T13:17:27.228 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:26 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] up:boot 2024-03-23T13:17:27.228 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:26 smithi007 ceph-mon[32184]: fsmap 8 up:standby 2024-03-23T13:17:27.228 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:26 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T13:17:27.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.641+0000 7fbf01e40640 1 -- 172.21.15.7:0/2999567892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbefc101760 msgr2=0x7fbefc101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.641+0000 7fbf01e40640 1 --2- 172.21.15.7:0/2999567892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbefc101760 0x7fbefc101b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009900 tx=0x7fbeec02bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.641+0000 7fbf01e40640 1 -- 172.21.15.7:0/2999567892 shutdown_connections 2024-03-23T13:17:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.641+0000 7fbf01e40640 1 --2- 172.21.15.7:0/2999567892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbefc103310 0x7fbefc109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.641+0000 7fbf01e40640 1 --2- 172.21.15.7:0/2999567892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc102950 0x7fbefc102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.641+0000 7fbf01e40640 1 --2- 172.21.15.7:0/2999567892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbefc101760 0x7fbefc101b80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.641+0000 7fbf01e40640 1 -- 172.21.15.7:0/2999567892 >> 172.21.15.7:0/2999567892 conn(0x7fbefc0fccb0 msgr2=0x7fbefc0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.641+0000 7fbf01e40640 1 -- 172.21.15.7:0/2999567892 shutdown_connections 2024-03-23T13:17:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.641+0000 7fbf01e40640 1 -- 172.21.15.7:0/2999567892 wait complete. 2024-03-23T13:17:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbf01e40640 1 Processor -- start 2024-03-23T13:17:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbf01e40640 1 -- start start 2024-03-23T13:17:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbf01e40640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbefc101760 0x7fbefc197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbf01e40640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc102950 0x7fbefc197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbf01e40640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbefc103310 0x7fbefc19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbf01e40640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc19f310 con 0x7fbefc101760 2024-03-23T13:17:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbf01e40640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc19f480 con 0x7fbefc103310 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbf01e40640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc19f640 con 0x7fbefc102950 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbefb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbefc101760 0x7fbefc197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbefb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbefc101760 0x7fbefc197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41604/0 (socket says 172.21.15.7:41604) 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.642+0000 7fbefb7fe640 1 -- 172.21.15.7:0/935645854 learned_addr learned my addr 172.21.15.7:0/935645854 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbefaffd640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc102950 0x7fbefc197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbefbfff640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbefc103310 0x7fbefc19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbefb7fe640 1 -- 172.21.15.7:0/935645854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc102950 msgr2=0x7fbefc197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbefb7fe640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc102950 0x7fbefc197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbefb7fe640 1 -- 172.21.15.7:0/935645854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbefc103310 msgr2=0x7fbefc19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbefb7fe640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbefc103310 0x7fbefc19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbefb7fe640 1 -- 172.21.15.7:0/935645854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec009560 con 0x7fbefc101760 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbefaffd640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc102950 0x7fbefc197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbefbfff640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbefc103310 0x7fbefc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:17:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbefb7fe640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbefc101760 0x7fbefc197820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbeec000c00 tx=0x7fbeec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:27.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbef8ff9640 1 -- 172.21.15.7:0/935645854 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004df0 con 0x7fbefc101760 2024-03-23T13:17:27.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbf01e40640 1 -- 172.21.15.7:0/935645854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbefc19f8c0 con 0x7fbefc101760 2024-03-23T13:17:27.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbef8ff9640 1 -- 172.21.15.7:0/935645854 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeec005430 con 0x7fbefc101760 2024-03-23T13:17:27.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbf01e40640 1 -- 172.21.15.7:0/935645854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbefc19fe20 con 0x7fbefc101760 2024-03-23T13:17:27.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.643+0000 7fbef8ff9640 1 -- 172.21.15.7:0/935645854 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec03e450 con 0x7fbefc101760 2024-03-23T13:17:27.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.644+0000 7fbf01e40640 1 -- 172.21.15.7:0/935645854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbefc108a80 con 0x7fbefc101760 2024-03-23T13:17:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.646+0000 7fbef8ff9640 1 -- 172.21.15.7:0/935645854 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fbeec005160 con 0x7fbefc101760 2024-03-23T13:17:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.646+0000 7fbef8ff9640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbee40a20c0 0x7fbee40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.646+0000 7fbefaffd640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbee40a20c0 0x7fbee40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.646+0000 7fbef8ff9640 1 -- 172.21.15.7:0/935645854 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbeec0f03d0 con 0x7fbefc101760 2024-03-23T13:17:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.647+0000 7fbefaffd640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbee40a20c0 0x7fbee40a4580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbefc198d30 tx=0x7fbef0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:27.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.653+0000 7fbef8ff9640 1 -- 172.21.15.7:0/935645854 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fbeec0b9840 con 0x7fbefc101760 2024-03-23T13:17:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.875+0000 7fbf01e40640 1 -- 172.21.15.7:0/935645854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbefc10d150 con 0x7fbefc101760 2024-03-23T13:17:27.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.876+0000 7fbef8ff9640 1 -- 172.21.15.7:0/935645854 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7fbeec0bcf00 con 0x7fbefc101760 2024-03-23T13:17:27.876 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-23T13:17:27.876 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-03-23T13:17:27.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 -- 172.21.15.7:0/935645854 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbee40a20c0 msgr2=0x7fbee40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:27.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbee40a20c0 0x7fbee40a4580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbefc198d30 tx=0x7fbef0005e50 comp rx=0 tx=0).stop 2024-03-23T13:17:27.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 -- 172.21.15.7:0/935645854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbefc101760 msgr2=0x7fbefc197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbefc101760 0x7fbefc197820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbeec000c00 tx=0x7fbeec004270 comp rx=0 tx=0).stop 2024-03-23T13:17:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 -- 172.21.15.7:0/935645854 shutdown_connections 2024-03-23T13:17:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbee40a20c0 0x7fbee40a4580 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbefc103310 0x7fbefc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc102950 0x7fbefc197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 --2- 172.21.15.7:0/935645854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbefc101760 0x7fbefc197820 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 -- 172.21.15.7:0/935645854 >> 172.21.15.7:0/935645854 conn(0x7fbefc0fccb0 msgr2=0x7fbefc105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 -- 172.21.15.7:0/935645854 shutdown_connections 2024-03-23T13:17:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:27.878+0000 7fbf01e40640 1 -- 172.21.15.7:0/935645854 wait complete. 2024-03-23T13:17:27.884 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:17:27 smithi155 systemd[1]: Starting Ceph mds.i for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:17:28.193 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:27 smithi130 ceph-mon[36225]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:28.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:27 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/935645854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:28.209 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:27 smithi155 ceph-mon[36157]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:28.209 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:27 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/935645854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:28.209 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:17:28 smithi155 podman[67152]: 2024-03-23 13:17:27.913193116 +0000 UTC m=+0.017632571 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:17:28.209 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:17:28 smithi155 podman[67152]: 2024-03-23 13:17:28.08766575 +0000 UTC m=+0.192105182 container create 255fec02aba7fb1e767d97f158eab740fcb4654f4c22802d00235a58ebbb90d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-23T13:17:28.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:27 smithi007 ceph-mon[32184]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:28.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:27 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/935645854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:28.537 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:17:28 smithi155 podman[67152]: 2024-03-23 13:17:28.298250764 +0000 UTC m=+0.402690203 container init 255fec02aba7fb1e767d97f158eab740fcb4654f4c22802d00235a58ebbb90d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-23T13:17:28.538 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:17:28 smithi155 podman[67152]: 2024-03-23 13:17:28.300867758 +0000 UTC m=+0.405307214 container start 255fec02aba7fb1e767d97f158eab740fcb4654f4c22802d00235a58ebbb90d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-23T13:17:28.538 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:17:28 smithi155 ceph-mds[67171]: -- 172.21.15.155:0/2172030117 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f52bd0c1a0 con 0x55f52bce2c00 2024-03-23T13:17:28.538 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:17:28 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i[67167]: starting mds.i at 2024-03-23T13:17:28.538 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:17:28 smithi155 bash[67152]: 255fec02aba7fb1e767d97f158eab740fcb4654f4c22802d00235a58ebbb90d6 2024-03-23T13:17:28.538 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:17:28 smithi155 systemd[1]: Started Ceph mds.i for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:29.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:29.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:29.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:29.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:29.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:29.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:29.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:29.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: Deploying daemon mds.g on smithi007 2024-03-23T13:17:29.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] up:boot 2024-03-23T13:17:29.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: fsmap 9 up:standby 2024-03-23T13:17:29.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:29 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T13:17:29.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:29.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:29.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:29.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:29.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:29.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:29.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:29.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: Deploying daemon mds.g on smithi007 2024-03-23T13:17:29.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] up:boot 2024-03-23T13:17:29.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: fsmap 9 up:standby 2024-03-23T13:17:29.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:29 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T13:17:29.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: Deploying daemon mds.g on smithi007 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] up:boot 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: fsmap 9 up:standby 2024-03-23T13:17:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:29 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T13:17:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.002+0000 7f8cef0eb640 1 -- 172.21.15.7:0/537219912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce8100650 msgr2=0x7f8ce8100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.002+0000 7f8cef0eb640 1 --2- 172.21.15.7:0/537219912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce8100650 0x7f8ce8100a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009900 tx=0x7f8cdc02bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.002+0000 7f8cef0eb640 1 -- 172.21.15.7:0/537219912 shutdown_connections 2024-03-23T13:17:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.002+0000 7f8cef0eb640 1 --2- 172.21.15.7:0/537219912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce8102200 0x7f8ce8108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.002+0000 7f8cef0eb640 1 --2- 172.21.15.7:0/537219912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce8101840 0x7f8ce8101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.002+0000 7f8cef0eb640 1 --2- 172.21.15.7:0/537219912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce8100650 0x7f8ce8100a70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.002+0000 7f8cef0eb640 1 -- 172.21.15.7:0/537219912 >> 172.21.15.7:0/537219912 conn(0x7f8ce80fbbc0 msgr2=0x7f8ce80fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.002+0000 7f8cef0eb640 1 -- 172.21.15.7:0/537219912 shutdown_connections 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.002+0000 7f8cef0eb640 1 -- 172.21.15.7:0/537219912 wait complete. 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cef0eb640 1 Processor -- start 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cef0eb640 1 -- start start 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cef0eb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce8100650 0x7f8ce8197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cef0eb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce8101840 0x7f8ce8197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cef0eb640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce8102200 0x7f8ce819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cef0eb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce819f320 con 0x7f8ce8100650 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cef0eb640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce819f490 con 0x7f8ce8102200 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cef0eb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce819f650 con 0x7f8ce8101840 2024-03-23T13:17:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cece87640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce8100650 0x7f8ce8197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cece87640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce8100650 0x7f8ce8197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41620/0 (socket says 172.21.15.7:41620) 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cece87640 1 -- 172.21.15.7:0/4119562521 learned_addr learned my addr 172.21.15.7:0/4119562521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8ce7fff640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce8101840 0x7f8ce8197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8ced688640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce8102200 0x7f8ce819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cece87640 1 -- 172.21.15.7:0/4119562521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce8101840 msgr2=0x7f8ce8197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cece87640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce8101840 0x7f8ce8197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cece87640 1 -- 172.21.15.7:0/4119562521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce8102200 msgr2=0x7f8ce819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cece87640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce8102200 0x7f8ce819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.003+0000 7f8cece87640 1 -- 172.21.15.7:0/4119562521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc009560 con 0x7f8ce8100650 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.004+0000 7f8cece87640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce8100650 0x7f8ce8197830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc000c00 tx=0x7f8cdc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.004+0000 7f8ce5ffb640 1 -- 172.21.15.7:0/4119562521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc004df0 con 0x7f8ce8100650 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.004+0000 7f8ce5ffb640 1 -- 172.21.15.7:0/4119562521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cdc005430 con 0x7f8ce8100650 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.004+0000 7f8cef0eb640 1 -- 172.21.15.7:0/4119562521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce819f8d0 con 0x7f8ce8100650 2024-03-23T13:17:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.004+0000 7f8ce5ffb640 1 -- 172.21.15.7:0/4119562521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc03e450 con 0x7f8ce8100650 2024-03-23T13:17:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.004+0000 7f8cef0eb640 1 -- 172.21.15.7:0/4119562521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce819fe30 con 0x7f8ce8100650 2024-03-23T13:17:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.005+0000 7f8cef0eb640 1 -- 172.21.15.7:0/4119562521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce810c060 con 0x7f8ce8100650 2024-03-23T13:17:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.006+0000 7f8ce5ffb640 1 -- 172.21.15.7:0/4119562521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f8cdc003680 con 0x7f8ce8100650 2024-03-23T13:17:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.006+0000 7f8ce5ffb640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd80a2140 0x7f8cd80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.007+0000 7f8ce5ffb640 1 -- 172.21.15.7:0/4119562521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8cdc0efe00 con 0x7f8ce8100650 2024-03-23T13:17:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.007+0000 7f8ce7fff640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd80a2140 0x7f8cd80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:30.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.007+0000 7f8ce7fff640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd80a2140 0x7f8cd80a4600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4005fd0 tx=0x7f8cd4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.011+0000 7f8ce5ffb640 1 -- 172.21.15.7:0/4119562521 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f8cdc0b91d0 con 0x7f8ce8100650 2024-03-23T13:17:30.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.233+0000 7f8cef0eb640 1 -- 172.21.15.7:0/4119562521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ce81a0290 con 0x7f8ce8100650 2024-03-23T13:17:30.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.234+0000 7f8ce5ffb640 1 -- 172.21.15.7:0/4119562521 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f8cdc0bc890 con 0x7f8ce8100650 2024-03-23T13:17:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:30.234 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-23T13:17:30.235 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 10 2024-03-23T13:17:30.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 -- 172.21.15.7:0/4119562521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd80a2140 msgr2=0x7f8cd80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:30.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd80a2140 0x7f8cd80a4600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4005fd0 tx=0x7f8cd4005850 comp rx=0 tx=0).stop 2024-03-23T13:17:30.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 -- 172.21.15.7:0/4119562521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce8100650 msgr2=0x7f8ce8197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:30.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce8100650 0x7f8ce8197830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc000c00 tx=0x7f8cdc004270 comp rx=0 tx=0).stop 2024-03-23T13:17:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 -- 172.21.15.7:0/4119562521 shutdown_connections 2024-03-23T13:17:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd80a2140 0x7f8cd80a4600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce8102200 0x7f8ce819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce8101840 0x7f8ce8197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 --2- 172.21.15.7:0/4119562521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce8100650 0x7f8ce8197830 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 -- 172.21.15.7:0/4119562521 >> 172.21.15.7:0/4119562521 conn(0x7f8ce80fbbc0 msgr2=0x7f8ce80fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 -- 172.21.15.7:0/4119562521 shutdown_connections 2024-03-23T13:17:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:30.236+0000 7f8cef0eb640 1 -- 172.21.15.7:0/4119562521 wait complete. 2024-03-23T13:17:30.559 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:30 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4119562521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:30.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:30 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/4119562521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:30.826 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:17:30 smithi007 systemd[1]: Starting Ceph mds.g for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:17:30.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:30 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/4119562521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:31.219 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:17:30 smithi007 podman[73395]: 2024-03-23 13:17:30.864099863 +0000 UTC m=+0.017615040 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:17:31.219 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:17:30 smithi007 podman[73395]: 2024-03-23 13:17:30.967562617 +0000 UTC m=+0.121077780 container create b0f5d7404803797bea7a4ff01baab27a6119f32ab6007eec0724650fe2670fc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:17:31.220 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:17:31 smithi007 podman[73395]: 2024-03-23 13:17:31.195259253 +0000 UTC m=+0.348774416 container init b0f5d7404803797bea7a4ff01baab27a6119f32ab6007eec0724650fe2670fc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-23T13:17:31.220 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:17:31 smithi007 podman[73395]: 2024-03-23 13:17:31.197712531 +0000 UTC m=+0.351227694 container start b0f5d7404803797bea7a4ff01baab27a6119f32ab6007eec0724650fe2670fc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-23T13:17:31.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:31 smithi007 ceph-mon[32184]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:31.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:31 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:31.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:31 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:31.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:31 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:31.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:31 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:31.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:31 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:31.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:31 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:31.577 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:17:31 smithi007 ceph-mds[73414]: -- 172.21.15.7:0/894615878 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d82ca961a0 con 0x55d82ca6d000 2024-03-23T13:17:31.577 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:17:31 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g[73410]: starting mds.g at 2024-03-23T13:17:31.577 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:17:31 smithi007 bash[73395]: b0f5d7404803797bea7a4ff01baab27a6119f32ab6007eec0724650fe2670fc1 2024-03-23T13:17:31.577 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:17:31 smithi007 systemd[1]: Started Ceph mds.g for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:31.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:31 smithi155 ceph-mon[36157]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:31.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:31 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:31.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:31 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:31.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:31 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:31.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:31 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:31.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:31 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:31.788 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:31 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:31.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:31 smithi130 ceph-mon[36225]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:31.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:31 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:31.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:31 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:31.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:31 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:31.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:31 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:31.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:31 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:31.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:31 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:32.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.358+0000 7f30c0247640 1 -- 172.21.15.7:0/2528782291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30b8104ad0 msgr2=0x7f30b8106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.358+0000 7f30c0247640 1 --2- 172.21.15.7:0/2528782291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30b8104ad0 0x7f30b8106ec0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009900 tx=0x7f30b402bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.358+0000 7f30c0247640 1 -- 172.21.15.7:0/2528782291 shutdown_connections 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.358+0000 7f30c0247640 1 --2- 172.21.15.7:0/2528782291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30b8107400 0x7f30b8109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.358+0000 7f30c0247640 1 --2- 172.21.15.7:0/2528782291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30b8104ad0 0x7f30b8106ec0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.358+0000 7f30c0247640 1 --2- 172.21.15.7:0/2528782291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30b81021a0 0x7f30b8104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.358+0000 7f30c0247640 1 -- 172.21.15.7:0/2528782291 >> 172.21.15.7:0/2528782291 conn(0x7f30b80fbb90 msgr2=0x7f30b80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.358+0000 7f30c0247640 1 -- 172.21.15.7:0/2528782291 shutdown_connections 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.358+0000 7f30c0247640 1 -- 172.21.15.7:0/2528782291 wait complete. 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30c0247640 1 Processor -- start 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30c0247640 1 -- start start 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30c0247640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30b81021a0 0x7f30b81955e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30c0247640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30b8104ad0 0x7f30b8195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30c0247640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30b8107400 0x7f30b819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30c0247640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30b819d0d0 con 0x7f30b81021a0 2024-03-23T13:17:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30c0247640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f30b819d240 con 0x7f30b8107400 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30c0247640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f30b819d400 con 0x7f30b8104ad0 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30bdfe3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30b81021a0 0x7f30b81955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30bdfe3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30b81021a0 0x7f30b81955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41670/0 (socket says 172.21.15.7:41670) 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30bdfe3640 1 -- 172.21.15.7:0/372963721 learned_addr learned my addr 172.21.15.7:0/372963721 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30bd7e2640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30b8104ad0 0x7f30b8195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30be7e4640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30b8107400 0x7f30b819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30bdfe3640 1 -- 172.21.15.7:0/372963721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30b8104ad0 msgr2=0x7f30b8195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30bdfe3640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30b8104ad0 0x7f30b8195b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30bdfe3640 1 -- 172.21.15.7:0/372963721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30b8107400 msgr2=0x7f30b819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30bdfe3640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30b8107400 0x7f30b819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30bdfe3640 1 -- 172.21.15.7:0/372963721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b4009560 con 0x7f30b81021a0 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30be7e4640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30b8107400 0x7f30b819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:17:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.359+0000 7f30bdfe3640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30b81021a0 0x7f30b81955e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f30a800d950 tx=0x7f30a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.360+0000 7f30aeffd640 1 -- 172.21.15.7:0/372963721 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8009c30 con 0x7f30b81021a0 2024-03-23T13:17:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.360+0000 7f30aeffd640 1 -- 172.21.15.7:0/372963721 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a8009d90 con 0x7f30b81021a0 2024-03-23T13:17:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.360+0000 7f30c0247640 1 -- 172.21.15.7:0/372963721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b819d6e0 con 0x7f30b81021a0 2024-03-23T13:17:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.360+0000 7f30aeffd640 1 -- 172.21.15.7:0/372963721 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a800f7e0 con 0x7f30b81021a0 2024-03-23T13:17:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.360+0000 7f30c0247640 1 -- 172.21.15.7:0/372963721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b819dbc0 con 0x7f30b81021a0 2024-03-23T13:17:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.361+0000 7f30c0247640 1 -- 172.21.15.7:0/372963721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b80fd680 con 0x7f30b81021a0 2024-03-23T13:17:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.362+0000 7f30aeffd640 1 -- 172.21.15.7:0/372963721 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f30a8010ad0 con 0x7f30b81021a0 2024-03-23T13:17:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.362+0000 7f30aeffd640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f30a40a64d0 0x7f30a40a8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.362+0000 7f30aeffd640 1 -- 172.21.15.7:0/372963721 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f30a80cd490 con 0x7f30b81021a0 2024-03-23T13:17:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.362+0000 7f30bd7e2640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f30a40a64d0 0x7f30a40a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.363+0000 7f30bd7e2640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f30a40a64d0 0x7f30a40a8990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f30b4005ab0 tx=0x7f30b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.367+0000 7f30aeffd640 1 -- 172.21.15.7:0/372963721 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f30a8096900 con 0x7f30b81021a0 2024-03-23T13:17:32.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:32 smithi007 ceph-mon[32184]: Deploying daemon mds.a on smithi007 2024-03-23T13:17:32.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:32 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.7:6838/388542752,v1:172.21.15.7:6839/388542752] up:boot 2024-03-23T13:17:32.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:32 smithi007 ceph-mon[32184]: fsmap 10 up:standby 2024-03-23T13:17:32.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:32 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T13:17:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.591+0000 7f30c0247640 1 -- 172.21.15.7:0/372963721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30b802d1d0 con 0x7f30b81021a0 2024-03-23T13:17:32.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.592+0000 7f30aeffd640 1 -- 172.21.15.7:0/372963721 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8821 (secure 0 0 0) 0x7f30a8099fc0 con 0x7f30b81021a0 2024-03-23T13:17:32.592 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:32.592 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-23T13:17:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 11 2024-03-23T13:17:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.594+0000 7f30c0247640 1 -- 172.21.15.7:0/372963721 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f30a40a64d0 msgr2=0x7f30a40a8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.594+0000 7f30c0247640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f30a40a64d0 0x7f30a40a8990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f30b4005ab0 tx=0x7f30b4005fd0 comp rx=0 tx=0).stop 2024-03-23T13:17:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.594+0000 7f30c0247640 1 -- 172.21.15.7:0/372963721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30b81021a0 msgr2=0x7f30b81955e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.594+0000 7f30c0247640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30b81021a0 0x7f30b81955e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f30a800d950 tx=0x7f30a800de10 comp rx=0 tx=0).stop 2024-03-23T13:17:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.594+0000 7f30c0247640 1 -- 172.21.15.7:0/372963721 shutdown_connections 2024-03-23T13:17:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.594+0000 7f30c0247640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f30a40a64d0 0x7f30a40a8990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.594+0000 7f30c0247640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30b8107400 0x7f30b819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.594+0000 7f30c0247640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30b8104ad0 0x7f30b8195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.594+0000 7f30c0247640 1 --2- 172.21.15.7:0/372963721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30b81021a0 0x7f30b81955e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.594+0000 7f30c0247640 1 -- 172.21.15.7:0/372963721 >> 172.21.15.7:0/372963721 conn(0x7f30b80fbb90 msgr2=0x7f30b8102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.595+0000 7f30c0247640 1 -- 172.21.15.7:0/372963721 shutdown_connections 2024-03-23T13:17:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:32.595+0000 7f30c0247640 1 -- 172.21.15.7:0/372963721 wait complete. 2024-03-23T13:17:32.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:32 smithi155 ceph-mon[36157]: Deploying daemon mds.a on smithi007 2024-03-23T13:17:32.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:32 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.7:6838/388542752,v1:172.21.15.7:6839/388542752] up:boot 2024-03-23T13:17:32.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:32 smithi155 ceph-mon[36157]: fsmap 10 up:standby 2024-03-23T13:17:32.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:32 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T13:17:32.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:32 smithi130 ceph-mon[36225]: Deploying daemon mds.a on smithi007 2024-03-23T13:17:32.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:32 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.7:6838/388542752,v1:172.21.15.7:6839/388542752] up:boot 2024-03-23T13:17:32.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:32 smithi130 ceph-mon[36225]: fsmap 10 up:standby 2024-03-23T13:17:32.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:32 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T13:17:33.755 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:33 smithi007 ceph-mon[32184]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:33.755 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:33 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/372963721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:33.755 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:17:33 smithi007 systemd[1]: Starting Ceph mds.a for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:17:33.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:33 smithi155 ceph-mon[36157]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:33.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:33 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/372963721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:33.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:33 smithi130 ceph-mon[36225]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:33.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:33 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/372963721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:34.008 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:17:33 smithi007 podman[73744]: 2024-03-23 13:17:33.659257253 +0000 UTC m=+0.018340372 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:17:34.008 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:17:33 smithi007 podman[73744]: 2024-03-23 13:17:33.773975466 +0000 UTC m=+0.133058563 container create 256ad5530e33c8f461e024fb0d6e7d47a4c27bbb261367f2a79b71c58a7259a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-23T13:17:34.008 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:17:34 smithi007 podman[73744]: 2024-03-23 13:17:34.010007399 +0000 UTC m=+0.369090514 container init 256ad5530e33c8f461e024fb0d6e7d47a4c27bbb261367f2a79b71c58a7259a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:17:34.326 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:17:34 smithi007 podman[73744]: 2024-03-23 13:17:34.012673448 +0000 UTC m=+0.371756546 container start 256ad5530e33c8f461e024fb0d6e7d47a4c27bbb261367f2a79b71c58a7259a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD) 2024-03-23T13:17:34.326 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:17:34 smithi007 ceph-mds[73763]: -- 172.21.15.7:0/2179211140 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a721a961a0 con 0x55a721a6d000 2024-03-23T13:17:34.326 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:17:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a[73759]: starting mds.a at 2024-03-23T13:17:34.327 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:17:34 smithi007 bash[73744]: 256ad5530e33c8f461e024fb0d6e7d47a4c27bbb261367f2a79b71c58a7259a6 2024-03-23T13:17:34.327 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:17:34 smithi007 systemd[1]: Started Ceph mds.a for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:34.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 -- 172.21.15.7:0/4261440671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618100630 msgr2=0x7f8618100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 --2- 172.21.15.7:0/4261440671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618100630 0x7f8618100a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f860c009900 tx=0x7f860c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 -- 172.21.15.7:0/4261440671 shutdown_connections 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 --2- 172.21.15.7:0/4261440671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86181021e0 0x7f8618108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 --2- 172.21.15.7:0/4261440671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8618101820 0x7f8618101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 --2- 172.21.15.7:0/4261440671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618100630 0x7f8618100a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 -- 172.21.15.7:0/4261440671 >> 172.21.15.7:0/4261440671 conn(0x7f86180fbba0 msgr2=0x7f86180fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 -- 172.21.15.7:0/4261440671 shutdown_connections 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 -- 172.21.15.7:0/4261440671 wait complete. 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 Processor -- start 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.718+0000 7f861ee23640 1 -- start start 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f861ee23640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8618100630 0x7f8618069400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f861ee23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618101820 0x7f861819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f861ee23640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86181021e0 0x7f861819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f861ee23640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f861819f2d0 con 0x7f8618101820 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f861ee23640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f861819f440 con 0x7f86181021e0 2024-03-23T13:17:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f861ee23640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f861819f600 con 0x7f8618100630 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8617fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618101820 0x7f861819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8617fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618101820 0x7f861819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35956/0 (socket says 172.21.15.7:35956) 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8617fff640 1 -- 172.21.15.7:0/2440160843 learned_addr learned my addr 172.21.15.7:0/2440160843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f861d3c0640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86181021e0 0x7f861819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f861cbbf640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8618100630 0x7f8618069400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8617fff640 1 -- 172.21.15.7:0/2440160843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8618100630 msgr2=0x7f8618069400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8617fff640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8618100630 0x7f8618069400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8617fff640 1 -- 172.21.15.7:0/2440160843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86181021e0 msgr2=0x7f861819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8617fff640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86181021e0 0x7f861819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8617fff640 1 -- 172.21.15.7:0/2440160843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860c009560 con 0x7f8618101820 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8617fff640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618101820 0x7f861819c500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f860400d950 tx=0x7f860400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8615ffb640 1 -- 172.21.15.7:0/2440160843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604009c30 con 0x7f8618101820 2024-03-23T13:17:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8615ffb640 1 -- 172.21.15.7:0/2440160843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8604009d90 con 0x7f8618101820 2024-03-23T13:17:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f861ee23640 1 -- 172.21.15.7:0/2440160843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861819f8e0 con 0x7f8618101820 2024-03-23T13:17:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.719+0000 7f8615ffb640 1 -- 172.21.15.7:0/2440160843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860400f7e0 con 0x7f8618101820 2024-03-23T13:17:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.720+0000 7f861ee23640 1 -- 172.21.15.7:0/2440160843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861819fdf0 con 0x7f8618101820 2024-03-23T13:17:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.720+0000 7f861ee23640 1 -- 172.21.15.7:0/2440160843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86181079d0 con 0x7f8618101820 2024-03-23T13:17:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.721+0000 7f8615ffb640 1 -- 172.21.15.7:0/2440160843 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f8604010ad0 con 0x7f8618101820 2024-03-23T13:17:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.722+0000 7f8615ffb640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f86080a20c0 0x7f86080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.722+0000 7f861cbbf640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f86080a20c0 0x7f86080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.722+0000 7f8615ffb640 1 -- 172.21.15.7:0/2440160843 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f86040cd5e0 con 0x7f8618101820 2024-03-23T13:17:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.722+0000 7f861cbbf640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f86080a20c0 0x7f86080a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f860c006010 tx=0x7f860c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.726+0000 7f8615ffb640 1 -- 172.21.15.7:0/2440160843 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f86040969b0 con 0x7f8618101820 2024-03-23T13:17:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.945+0000 7f861ee23640 1 -- 172.21.15.7:0/2440160843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f861802d1d0 con 0x7f8618101820 2024-03-23T13:17:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.945+0000 7f8615ffb640 1 -- 172.21.15.7:0/2440160843 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7f860409a070 con 0x7f8618101820 2024-03-23T13:17:34.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:34.946 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3415570481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3415570481},{"type":"v1","addr":"172.21.15.7:6841","nonce":3415570481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-23T13:17:34.946 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 -- 172.21.15.7:0/2440160843 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f86080a20c0 msgr2=0x7f86080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f86080a20c0 0x7f86080a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f860c006010 tx=0x7f860c000bc0 comp rx=0 tx=0).stop 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 -- 172.21.15.7:0/2440160843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618101820 msgr2=0x7f861819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618101820 0x7f861819c500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f860400d950 tx=0x7f860400de10 comp rx=0 tx=0).stop 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 -- 172.21.15.7:0/2440160843 shutdown_connections 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f86080a20c0 0x7f86080a4580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86181021e0 0x7f861819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618101820 0x7f861819c500 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 --2- 172.21.15.7:0/2440160843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8618100630 0x7f8618069400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 -- 172.21.15.7:0/2440160843 >> 172.21.15.7:0/2440160843 conn(0x7f86180fbba0 msgr2=0x7f8618104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 -- 172.21.15.7:0/2440160843 shutdown_connections 2024-03-23T13:17:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:34.947+0000 7f861ee23640 1 -- 172.21.15.7:0/2440160843 wait complete. 2024-03-23T13:17:35.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:35.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:35.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:35.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:35.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:35.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:35.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:35.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: Deploying daemon mds.b on smithi130 2024-03-23T13:17:35.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.7:6840/3415570481,v1:172.21.15.7:6841/3415570481] up:boot 2024-03-23T13:17:35.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: fsmap 11 up:standby 2024-03-23T13:17:35.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T13:17:35.327 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:35 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2440160843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: Deploying daemon mds.b on smithi130 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.7:6840/3415570481,v1:172.21.15.7:6841/3415570481] up:boot 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: fsmap 11 up:standby 2024-03-23T13:17:35.360 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T13:17:35.361 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:35 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2440160843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:35.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: Deploying daemon mds.b on smithi130 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.7:6840/3415570481,v1:172.21.15.7:6841/3415570481] up:boot 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: fsmap 11 up:standby 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T13:17:35.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:35 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2440160843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:36.684 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:36 smithi130 systemd[1]: Starting Ceph mds.b for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:17:36.944 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:36 smithi130 podman[67018]: 2024-03-23 13:17:36.588061094 +0000 UTC m=+0.017964328 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:17:36.944 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:36 smithi130 podman[67018]: 2024-03-23 13:17:36.723777034 +0000 UTC m=+0.153680253 container create 4fde6ee0de3a642b8ca2f9b83225217aec277c24529e913b61689b980196ecd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-23T13:17:36.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:37.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.078+0000 7f31a788f640 1 -- 172.21.15.7:0/2098122893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a01006b0 msgr2=0x7f31a0100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.078+0000 7f31a788f640 1 --2- 172.21.15.7:0/2098122893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a01006b0 0x7f31a0100ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f319c009900 tx=0x7f319c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.078+0000 7f31a788f640 1 -- 172.21.15.7:0/2098122893 shutdown_connections 2024-03-23T13:17:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.078+0000 7f31a788f640 1 --2- 172.21.15.7:0/2098122893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a0102260 0x7f31a0108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.078+0000 7f31a788f640 1 --2- 172.21.15.7:0/2098122893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31a01018a0 0x7f31a0101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.078+0000 7f31a788f640 1 --2- 172.21.15.7:0/2098122893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a01006b0 0x7f31a0100ad0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.078+0000 7f31a788f640 1 -- 172.21.15.7:0/2098122893 >> 172.21.15.7:0/2098122893 conn(0x7f31a00fbc40 msgr2=0x7f31a00fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.078+0000 7f31a788f640 1 -- 172.21.15.7:0/2098122893 shutdown_connections 2024-03-23T13:17:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.078+0000 7f31a788f640 1 -- 172.21.15.7:0/2098122893 wait complete. 2024-03-23T13:17:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a788f640 1 Processor -- start 2024-03-23T13:17:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a788f640 1 -- start start 2024-03-23T13:17:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a788f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a01006b0 0x7f31a0104710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a788f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31a01018a0 0x7f31a0102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a788f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a0102260 0x7f31a0103290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a788f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01039f0 con 0x7f31a01006b0 2024-03-23T13:17:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a788f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0103b60 con 0x7f31a01018a0 2024-03-23T13:17:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a788f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01a1840 con 0x7f31a0102260 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a562b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a01006b0 0x7f31a0104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a562b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a01006b0 0x7f31a0104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35964/0 (socket says 172.21.15.7:35964) 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a562b640 1 -- 172.21.15.7:0/4276854815 learned_addr learned my addr 172.21.15.7:0/4276854815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a4e2a640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31a01018a0 0x7f31a0102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a562b640 1 -- 172.21.15.7:0/4276854815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a0102260 msgr2=0x7f31a0103290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a5e2c640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a0102260 0x7f31a0103290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a562b640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a0102260 0x7f31a0103290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a562b640 1 -- 172.21.15.7:0/4276854815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31a01018a0 msgr2=0x7f31a0102d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a562b640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31a01018a0 0x7f31a0102d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.079+0000 7f31a562b640 1 -- 172.21.15.7:0/4276854815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c009560 con 0x7f31a01006b0 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.080+0000 7f31a562b640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a01006b0 0x7f31a0104710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f319c000c00 tx=0x7f319c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.080+0000 7f31967fc640 1 -- 172.21.15.7:0/4276854815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c004aa0 con 0x7f31a01006b0 2024-03-23T13:17:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.080+0000 7f31967fc640 1 -- 172.21.15.7:0/4276854815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f319c0052d0 con 0x7f31a01006b0 2024-03-23T13:17:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.080+0000 7f31a788f640 1 -- 172.21.15.7:0/4276854815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a01a1a90 con 0x7f31a01006b0 2024-03-23T13:17:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.080+0000 7f31967fc640 1 -- 172.21.15.7:0/4276854815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c003e60 con 0x7f31a01006b0 2024-03-23T13:17:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.080+0000 7f31a788f640 1 -- 172.21.15.7:0/4276854815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a01a1fa0 con 0x7f31a01006b0 2024-03-23T13:17:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.081+0000 7f31a788f640 1 -- 172.21.15.7:0/4276854815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a010c050 con 0x7f31a01006b0 2024-03-23T13:17:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.082+0000 7f31967fc640 1 -- 172.21.15.7:0/4276854815 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f319c005440 con 0x7f31a01006b0 2024-03-23T13:17:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.082+0000 7f31967fc640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f318c0a20c0 0x7f318c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.083+0000 7f31a4e2a640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f318c0a20c0 0x7f318c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.083+0000 7f31967fc640 1 -- 172.21.15.7:0/4276854815 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f319c0efdf0 con 0x7f31a01006b0 2024-03-23T13:17:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.083+0000 7f31a4e2a640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f318c0a20c0 0x7f318c0a4580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31a0104480 tx=0x7f3190005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.087+0000 7f31967fc640 1 -- 172.21.15.7:0/4276854815 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f319c0b91c0 con 0x7f31a01006b0 2024-03-23T13:17:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.310+0000 7f31a788f640 1 -- 172.21.15.7:0/4276854815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31a010c1e0 con 0x7f31a01006b0 2024-03-23T13:17:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.311+0000 7f31967fc640 1 -- 172.21.15.7:0/4276854815 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7f319c0bc880 con 0x7f31a01006b0 2024-03-23T13:17:37.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:37.311 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3415570481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3415570481},{"type":"v1","addr":"172.21.15.7:6841","nonce":3415570481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/791500055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":791500055},{"type":"v1","addr":"172.21.15.130:6839","nonce":791500055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-23T13:17:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2024-03-23T13:17:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 -- 172.21.15.7:0/4276854815 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f318c0a20c0 msgr2=0x7f318c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f318c0a20c0 0x7f318c0a4580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31a0104480 tx=0x7f3190005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 -- 172.21.15.7:0/4276854815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a01006b0 msgr2=0x7f31a0104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a01006b0 0x7f31a0104710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f319c000c00 tx=0x7f319c004270 comp rx=0 tx=0).stop 2024-03-23T13:17:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 -- 172.21.15.7:0/4276854815 shutdown_connections 2024-03-23T13:17:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f318c0a20c0 0x7f318c0a4580 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a0102260 0x7f31a0103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f31a01018a0 0x7f31a0102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 --2- 172.21.15.7:0/4276854815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a01006b0 0x7f31a0104710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 -- 172.21.15.7:0/4276854815 >> 172.21.15.7:0/4276854815 conn(0x7f31a00fbc40 msgr2=0x7f31a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:37.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 -- 172.21.15.7:0/4276854815 shutdown_connections 2024-03-23T13:17:37.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.313+0000 7f31a788f640 1 -- 172.21.15.7:0/4276854815 wait complete. 2024-03-23T13:17:37.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:37 smithi007 ceph-mon[32184]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:37.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.324 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:37 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.325 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:17:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 -- 172.21.15.7:0/3382884544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b41021e0 msgr2=0x7f63b4108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 --2- 172.21.15.7:0/3382884544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b41021e0 0x7f63b4108a70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009ca0 tx=0x7f63ac02c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 -- 172.21.15.7:0/3382884544 shutdown_connections 2024-03-23T13:17:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 --2- 172.21.15.7:0/3382884544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b41021e0 0x7f63b4108a70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 --2- 172.21.15.7:0/3382884544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63b4101820 0x7f63b4101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 --2- 172.21.15.7:0/3382884544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63b4100630 0x7f63b4100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 -- 172.21.15.7:0/3382884544 >> 172.21.15.7:0/3382884544 conn(0x7f63b40fbba0 msgr2=0x7f63b40fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 -- 172.21.15.7:0/3382884544 shutdown_connections 2024-03-23T13:17:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 -- 172.21.15.7:0/3382884544 wait complete. 2024-03-23T13:17:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 Processor -- start 2024-03-23T13:17:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.430+0000 7f63bb855640 1 -- start start 2024-03-23T13:17:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63bb855640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63b4100630 0x7f63b4197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63bb855640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b4101820 0x7f63b4197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63bb855640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63b41021e0 0x7f63b419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63bb855640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63b419f270 con 0x7f63b4101820 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63bb855640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f63b419f3e0 con 0x7f63b41021e0 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63bb855640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63b419f5a0 con 0x7f63b4100630 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b8df0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b4101820 0x7f63b4197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b8df0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b4101820 0x7f63b4197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35976/0 (socket says 172.21.15.7:35976) 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b8df0640 1 -- 172.21.15.7:0/2805703320 learned_addr learned my addr 172.21.15.7:0/2805703320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b95f1640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63b4100630 0x7f63b4197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b9df2640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63b41021e0 0x7f63b419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b8df0640 1 -- 172.21.15.7:0/2805703320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63b4100630 msgr2=0x7f63b4197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b8df0640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63b4100630 0x7f63b4197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b8df0640 1 -- 172.21.15.7:0/2805703320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63b41021e0 msgr2=0x7f63b419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b8df0640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63b41021e0 0x7f63b419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b8df0640 1 -- 172.21.15.7:0/2805703320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac009900 con 0x7f63b4101820 2024-03-23T13:17:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63b8df0640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b4101820 0x7f63b4197cc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63a400e9f0 tx=0x7f63a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63aa7fc640 1 -- 172.21.15.7:0/2805703320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4009700 con 0x7f63b4101820 2024-03-23T13:17:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63aa7fc640 1 -- 172.21.15.7:0/2805703320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a400cd80 con 0x7f63b4101820 2024-03-23T13:17:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63bb855640 1 -- 172.21.15.7:0/2805703320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b419f880 con 0x7f63b4101820 2024-03-23T13:17:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63aa7fc640 1 -- 172.21.15.7:0/2805703320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4018920 con 0x7f63b4101820 2024-03-23T13:17:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.431+0000 7f63bb855640 1 -- 172.21.15.7:0/2805703320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b419fde0 con 0x7f63b4101820 2024-03-23T13:17:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.433+0000 7f63bb855640 1 -- 172.21.15.7:0/2805703320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b410c010 con 0x7f63b4101820 2024-03-23T13:17:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.434+0000 7f63aa7fc640 1 -- 172.21.15.7:0/2805703320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f63a4018a80 con 0x7f63b4101820 2024-03-23T13:17:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.435+0000 7f63aa7fc640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f63a00a2140 0x7f63a00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.435+0000 7f63b95f1640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f63a00a2140 0x7f63a00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.435+0000 7f63aa7fc640 1 -- 172.21.15.7:0/2805703320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f63a4014070 con 0x7f63b4101820 2024-03-23T13:17:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.435+0000 7f63b95f1640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f63a00a2140 0x7f63a00a4600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005fd0 tx=0x7f63b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.440+0000 7f63aa7fc640 1 -- 172.21.15.7:0/2805703320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f63a4053280 con 0x7f63b4101820 2024-03-23T13:17:37.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:37 smithi130 ceph-mon[36225]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:37.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:37 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.444 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:36 smithi130 podman[67018]: 2024-03-23 13:17:36.989384492 +0000 UTC m=+0.419287711 container init 4fde6ee0de3a642b8ca2f9b83225217aec277c24529e913b61689b980196ecd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-23T13:17:37.444 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:36 smithi130 podman[67018]: 2024-03-23 13:17:36.992253643 +0000 UTC m=+0.422156862 container start 4fde6ee0de3a642b8ca2f9b83225217aec277c24529e913b61689b980196ecd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-03-23T13:17:37.444 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:37 smithi130 ceph-mds[67037]: -- 172.21.15.130:0/1521285237 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bff244e1a0 con 0x55bff2424c00 2024-03-23T13:17:37.445 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:37 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: starting mds.b at 2024-03-23T13:17:37.445 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:37 smithi130 bash[67018]: 4fde6ee0de3a642b8ca2f9b83225217aec277c24529e913b61689b980196ecd7 2024-03-23T13:17:37.445 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:37 smithi130 systemd[1]: Started Ceph mds.b for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:17:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:37 smithi155 ceph-mon[36157]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:37 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.624+0000 7f63bb855640 1 -- 172.21.15.7:0/2805703320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63b41a0210 con 0x7f63b4101820 2024-03-23T13:17:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.625+0000 7f63aa7fc640 1 -- 172.21.15.7:0/2805703320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18361 (secure 0 0 0) 0x7f63a4096f90 con 0x7f63b4101820 2024-03-23T13:17:37.626 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:37.626 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:16:54.558135+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.627+0000 7f63bb855640 1 -- 172.21.15.7:0/2805703320 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f63a00a2140 msgr2=0x7f63a00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.627+0000 7f63bb855640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f63a00a2140 0x7f63a00a4600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005fd0 tx=0x7f63b000b410 comp rx=0 tx=0).stop 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.627+0000 7f63bb855640 1 -- 172.21.15.7:0/2805703320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b4101820 msgr2=0x7f63b4197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.627+0000 7f63bb855640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b4101820 0x7f63b4197cc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63a400e9f0 tx=0x7f63a400eeb0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.627+0000 7f63bb855640 1 -- 172.21.15.7:0/2805703320 shutdown_connections 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.627+0000 7f63bb855640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f63a00a2140 0x7f63a00a4600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.627+0000 7f63bb855640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63b41021e0 0x7f63b419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.627+0000 7f63bb855640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b4101820 0x7f63b4197cc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.627+0000 7f63bb855640 1 --2- 172.21.15.7:0/2805703320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63b4100630 0x7f63b4197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.627+0000 7f63bb855640 1 -- 172.21.15.7:0/2805703320 >> 172.21.15.7:0/2805703320 conn(0x7f63b40fbba0 msgr2=0x7f63b40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.628+0000 7f63bb855640 1 -- 172.21.15.7:0/2805703320 shutdown_connections 2024-03-23T13:17:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.628+0000 7f63bb855640 1 -- 172.21.15.7:0/2805703320 wait complete. 2024-03-23T13:17:37.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T13:13:08.186719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T13:17:37.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T13:17:37.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.745+0000 7fabb5c88640 1 -- 172.21.15.7:0/4040742454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb01021c0 msgr2=0x7fabb0108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.745+0000 7fabb5c88640 1 --2- 172.21.15.7:0/4040742454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb01021c0 0x7fabb0108a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faba8009ca0 tx=0x7faba802c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.745+0000 7fabb5c88640 1 -- 172.21.15.7:0/4040742454 shutdown_connections 2024-03-23T13:17:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.745+0000 7fabb5c88640 1 --2- 172.21.15.7:0/4040742454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb01021c0 0x7fabb0108a50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.745+0000 7fabb5c88640 1 --2- 172.21.15.7:0/4040742454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb0101800 0x7fabb0101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.745+0000 7fabb5c88640 1 --2- 172.21.15.7:0/4040742454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb0100610 0x7fabb0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.745+0000 7fabb5c88640 1 -- 172.21.15.7:0/4040742454 >> 172.21.15.7:0/4040742454 conn(0x7fabb00fbb80 msgr2=0x7fabb00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.745+0000 7fabb5c88640 1 -- 172.21.15.7:0/4040742454 shutdown_connections 2024-03-23T13:17:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.745+0000 7fabb5c88640 1 -- 172.21.15.7:0/4040742454 wait complete. 2024-03-23T13:17:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabb5c88640 1 Processor -- start 2024-03-23T13:17:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabb5c88640 1 -- start start 2024-03-23T13:17:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabb5c88640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb0100610 0x7fabb01977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabb5c88640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb0101800 0x7fabb0197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabb5c88640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb01021c0 0x7fabb019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabb5c88640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fabb019f260 con 0x7fabb0101800 2024-03-23T13:17:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabb5c88640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fabb019f3d0 con 0x7fabb0100610 2024-03-23T13:17:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabb5c88640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fabb019f590 con 0x7fabb01021c0 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabaeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb0101800 0x7fabb0197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabaeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb0101800 0x7fabb0197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35984/0 (socket says 172.21.15.7:35984) 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabaeffd640 1 -- 172.21.15.7:0/1802593365 learned_addr learned my addr 172.21.15.7:0/1802593365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabaffff640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb01021c0 0x7fabb019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.746+0000 7fabaf7fe640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb0100610 0x7fabb01977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabaeffd640 1 -- 172.21.15.7:0/1802593365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb01021c0 msgr2=0x7fabb019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabaeffd640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb01021c0 0x7fabb019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabaeffd640 1 -- 172.21.15.7:0/1802593365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb0100610 msgr2=0x7fabb01977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabaeffd640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb0100610 0x7fabb01977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabaeffd640 1 -- 172.21.15.7:0/1802593365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba8009900 con 0x7fabb0101800 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabaeffd640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb0101800 0x7fabb0197d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faba400e9f0 tx=0x7faba400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabacff9640 1 -- 172.21.15.7:0/1802593365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4009720 con 0x7fabb0101800 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabacff9640 1 -- 172.21.15.7:0/1802593365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba400cd80 con 0x7fabb0101800 2024-03-23T13:17:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabb5c88640 1 -- 172.21.15.7:0/1802593365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb019f870 con 0x7fabb0101800 2024-03-23T13:17:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabacff9640 1 -- 172.21.15.7:0/1802593365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4010690 con 0x7fabb0101800 2024-03-23T13:17:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.747+0000 7fabb5c88640 1 -- 172.21.15.7:0/1802593365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb019fd30 con 0x7fabb0101800 2024-03-23T13:17:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.748+0000 7fabb5c88640 1 -- 172.21.15.7:0/1802593365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb010c020 con 0x7fabb0101800 2024-03-23T13:17:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.750+0000 7fabacff9640 1 -- 172.21.15.7:0/1802593365 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7faba40046c0 con 0x7fabb0101800 2024-03-23T13:17:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.750+0000 7fabacff9640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fab980a20c0 0x7fab980a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.750+0000 7fabaf7fe640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fab980a20c0 0x7fab980a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.751+0000 7fabacff9640 1 -- 172.21.15.7:0/1802593365 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7faba4014070 con 0x7fabb0101800 2024-03-23T13:17:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.751+0000 7fabaf7fe640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fab980a20c0 0x7fab980a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faba0005fd0 tx=0x7faba000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:37.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.754+0000 7fabacff9640 1 -- 172.21.15.7:0/1802593365 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7faba4097140 con 0x7fabb0101800 2024-03-23T13:17:37.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.957+0000 7fabb5c88640 1 -- 172.21.15.7:0/1802593365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fabb019ffb0 con 0x7fabb0101800 2024-03-23T13:17:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.958+0000 7fabacff9640 1 -- 172.21.15.7:0/1802593365 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7faba409a800 con 0x7fabb0101800 2024-03-23T13:17:37.958 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-23T13:17:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 -- 172.21.15.7:0/1802593365 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fab980a20c0 msgr2=0x7fab980a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fab980a20c0 0x7fab980a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faba0005fd0 tx=0x7faba000b410 comp rx=0 tx=0).stop 2024-03-23T13:17:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 -- 172.21.15.7:0/1802593365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb0101800 msgr2=0x7fabb0197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb0101800 0x7fabb0197d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faba400e9f0 tx=0x7faba400edf0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 -- 172.21.15.7:0/1802593365 shutdown_connections 2024-03-23T13:17:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fab980a20c0 0x7fab980a4580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb01021c0 0x7fabb019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb0101800 0x7fabb0197d00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 --2- 172.21.15.7:0/1802593365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb0100610 0x7fabb01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 -- 172.21.15.7:0/1802593365 >> 172.21.15.7:0/1802593365 conn(0x7fabb00fbb80 msgr2=0x7fabb00fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 -- 172.21.15.7:0/1802593365 shutdown_connections 2024-03-23T13:17:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:37.960+0000 7fabb5c88640 1 -- 172.21.15.7:0/1802593365 wait complete. 2024-03-23T13:17:37.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T13:17:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.074+0000 7f36b21b6640 1 -- 172.21.15.7:0/3033120958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac102910 msgr2=0x7f36ac102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.074+0000 7f36b21b6640 1 --2- 172.21.15.7:0/3033120958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac102910 0x7f36ac102d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3694009900 tx=0x7f369402bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.074+0000 7f36b21b6640 1 -- 172.21.15.7:0/3033120958 shutdown_connections 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.074+0000 7f36b21b6640 1 --2- 172.21.15.7:0/3033120958 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36ac1032d0 0x7f36ac109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.074+0000 7f36b21b6640 1 --2- 172.21.15.7:0/3033120958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac102910 0x7f36ac102d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.074+0000 7f36b21b6640 1 --2- 172.21.15.7:0/3033120958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac101720 0x7f36ac101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.074+0000 7f36b21b6640 1 -- 172.21.15.7:0/3033120958 >> 172.21.15.7:0/3033120958 conn(0x7f36ac0fccb0 msgr2=0x7f36ac0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.074+0000 7f36b21b6640 1 -- 172.21.15.7:0/3033120958 shutdown_connections 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.074+0000 7f36b21b6640 1 -- 172.21.15.7:0/3033120958 wait complete. 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36b21b6640 1 Processor -- start 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36b21b6640 1 -- start start 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36b21b6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36ac101720 0x7f36ac1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36b21b6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac102910 0x7f36ac197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36b21b6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac1032d0 0x7f36ac19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36b21b6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac19f2a0 con 0x7f36ac1032d0 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36b21b6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac19f410 con 0x7f36ac101720 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36b21b6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac19f5d0 con 0x7f36ac102910 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36abfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac1032d0 0x7f36ac19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36abfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac1032d0 0x7f36ac19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36004/0 (socket says 172.21.15.7:36004) 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36abfff640 1 -- 172.21.15.7:0/3037485274 learned_addr learned my addr 172.21.15.7:0/3037485274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36ab7fe640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36ac101720 0x7f36ac1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36a3fff640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac102910 0x7f36ac197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36abfff640 1 -- 172.21.15.7:0/3037485274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac102910 msgr2=0x7f36ac197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36abfff640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac102910 0x7f36ac197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36abfff640 1 -- 172.21.15.7:0/3037485274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36ac101720 msgr2=0x7f36ac1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36abfff640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36ac101720 0x7f36ac1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36abfff640 1 -- 172.21.15.7:0/3037485274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3694009560 con 0x7f36ac1032d0 2024-03-23T13:17:38.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.075+0000 7f36abfff640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac1032d0 0x7f36ac19ed60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f36a400eb10 tx=0x7f36a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:38.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.076+0000 7f36a97fa640 1 -- 172.21.15.7:0/3037485274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4004b20 con 0x7f36ac1032d0 2024-03-23T13:17:38.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.076+0000 7f36a97fa640 1 -- 172.21.15.7:0/3037485274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a4004c80 con 0x7f36ac1032d0 2024-03-23T13:17:38.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.076+0000 7f36b21b6640 1 -- 172.21.15.7:0/3037485274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ac19f8b0 con 0x7f36ac1032d0 2024-03-23T13:17:38.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.076+0000 7f36a97fa640 1 -- 172.21.15.7:0/3037485274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4010650 con 0x7f36ac1032d0 2024-03-23T13:17:38.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.076+0000 7f36b21b6640 1 -- 172.21.15.7:0/3037485274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ac19fdc0 con 0x7f36ac1032d0 2024-03-23T13:17:38.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.077+0000 7f36b21b6640 1 -- 172.21.15.7:0/3037485274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ac108a40 con 0x7f36ac1032d0 2024-03-23T13:17:38.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.078+0000 7f36a97fa640 1 -- 172.21.15.7:0/3037485274 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f36a40107f0 con 0x7f36ac1032d0 2024-03-23T13:17:38.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.078+0000 7f36a97fa640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f36980a2140 0x7f36980a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:38.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.078+0000 7f36a97fa640 1 -- 172.21.15.7:0/3037485274 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f36a4014070 con 0x7f36ac1032d0 2024-03-23T13:17:38.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.078+0000 7f36ab7fe640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f36980a2140 0x7f36980a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:38.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.079+0000 7f36ab7fe640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f36980a2140 0x7f36980a4600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f369c005fd0 tx=0x7f369c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:38.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.082+0000 7f36a97fa640 1 -- 172.21.15.7:0/3037485274 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f36a4097430 con 0x7f36ac1032d0 2024-03-23T13:17:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.269+0000 7f36b21b6640 1 -- 172.21.15.7:0/3037485274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f36ac10d130 con 0x7f36ac1032d0 2024-03-23T13:17:38.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.270+0000 7f36a97fa640 1 -- 172.21.15.7:0/3037485274 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f36a409aaf0 con 0x7f36ac1032d0 2024-03-23T13:17:38.270 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:38.270 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-03-23T13:17:38.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:17:38.270 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-03-23T13:17:38.271 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-03-23T13:17:38.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:17:38.271 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-03-23T13:17:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.271+0000 7f36b21b6640 1 -- 172.21.15.7:0/3037485274 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f36980a2140 msgr2=0x7f36980a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f36980a2140 0x7f36980a4600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f369c005fd0 tx=0x7f369c00b390 comp rx=0 tx=0).stop 2024-03-23T13:17:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 -- 172.21.15.7:0/3037485274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac1032d0 msgr2=0x7f36ac19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac1032d0 0x7f36ac19ed60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f36a400eb10 tx=0x7f36a400c2d0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 -- 172.21.15.7:0/3037485274 shutdown_connections 2024-03-23T13:17:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f36980a2140 0x7f36980a4600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac1032d0 0x7f36ac19ed60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ac102910 0x7f36ac197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 --2- 172.21.15.7:0/3037485274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f36ac101720 0x7f36ac1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 -- 172.21.15.7:0/3037485274 >> 172.21.15.7:0/3037485274 conn(0x7f36ac0fccb0 msgr2=0x7f36ac105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 -- 172.21.15.7:0/3037485274 shutdown_connections 2024-03-23T13:17:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.272+0000 7f36b21b6640 1 -- 172.21.15.7:0/3037485274 wait complete. 2024-03-23T13:17:38.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:38 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] up:boot 2024-03-23T13:17:38.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:38 smithi007 ceph-mon[32184]: fsmap 12 up:standby 2024-03-23T13:17:38.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:38 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T13:17:38.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:38 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4276854815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:38.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:38 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:17:38.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:38 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2805703320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:38.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:38 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1802593365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:17:38.284 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T13:17:38.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T13:17:38.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 -- 172.21.15.7:0/1249570083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c8100650 msgr2=0x7f23c8100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:38.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 --2- 172.21.15.7:0/1249570083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c8100650 0x7f23c8100a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009900 tx=0x7f23bc02bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:38.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 -- 172.21.15.7:0/1249570083 shutdown_connections 2024-03-23T13:17:38.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 --2- 172.21.15.7:0/1249570083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8102200 0x7f23c8108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 --2- 172.21.15.7:0/1249570083 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23c8101840 0x7f23c8101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 --2- 172.21.15.7:0/1249570083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c8100650 0x7f23c8100a70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 -- 172.21.15.7:0/1249570083 >> 172.21.15.7:0/1249570083 conn(0x7f23c80fbbc0 msgr2=0x7f23c80fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 -- 172.21.15.7:0/1249570083 shutdown_connections 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 -- 172.21.15.7:0/1249570083 wait complete. 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 Processor -- start 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 -- start start 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c8100650 0x7f23c8196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8101840 0x7f23c8196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23c8102200 0x7f23c819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:38.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23c819e280 con 0x7f23c8100650 2024-03-23T13:17:38.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23c819e3f0 con 0x7f23c8102200 2024-03-23T13:17:38.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23ceadf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23c819e5b0 con 0x7f23c8101840 2024-03-23T13:17:38.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23cc87b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c8100650 0x7f23c8196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:38.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23cc87b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c8100650 0x7f23c8196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36022/0 (socket says 172.21.15.7:36022) 2024-03-23T13:17:38.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.392+0000 7f23cc87b640 1 -- 172.21.15.7:0/1271342328 learned_addr learned my addr 172.21.15.7:0/1271342328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:38.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23cc87b640 1 -- 172.21.15.7:0/1271342328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8101840 msgr2=0x7f23c8196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:38.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23cc87b640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8101840 0x7f23c8196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23cc87b640 1 -- 172.21.15.7:0/1271342328 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23c8102200 msgr2=0x7f23c819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:38.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23cc87b640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23c8102200 0x7f23c819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:38.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23cc87b640 1 -- 172.21.15.7:0/1271342328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc009560 con 0x7f23c8100650 2024-03-23T13:17:38.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23cc87b640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c8100650 0x7f23c8196790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23bc005ab0 tx=0x7f23bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:38.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23c5ffb640 1 -- 172.21.15.7:0/1271342328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc004a80 con 0x7f23c8100650 2024-03-23T13:17:38.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23c5ffb640 1 -- 172.21.15.7:0/1271342328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23bc0052d0 con 0x7f23c8100650 2024-03-23T13:17:38.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23ceadf640 1 -- 172.21.15.7:0/1271342328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c819e830 con 0x7f23c8100650 2024-03-23T13:17:38.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23c5ffb640 1 -- 172.21.15.7:0/1271342328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc003e50 con 0x7f23c8100650 2024-03-23T13:17:38.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.393+0000 7f23ceadf640 1 -- 172.21.15.7:0/1271342328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c819ed40 con 0x7f23c8100650 2024-03-23T13:17:38.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.394+0000 7f23ceadf640 1 -- 172.21.15.7:0/1271342328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c8107a30 con 0x7f23c8100650 2024-03-23T13:17:38.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.395+0000 7f23c5ffb640 1 -- 172.21.15.7:0/1271342328 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f23bc005440 con 0x7f23c8100650 2024-03-23T13:17:38.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.396+0000 7f23c5ffb640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f23b80a20c0 0x7f23b80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:38.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.396+0000 7f23c7fff640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f23b80a20c0 0x7f23b80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:38.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.396+0000 7f23c5ffb640 1 -- 172.21.15.7:0/1271342328 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f23bc0efd70 con 0x7f23c8100650 2024-03-23T13:17:38.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.396+0000 7f23c7fff640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f23b80a20c0 0x7f23b80a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005fd0 tx=0x7f23b4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:38.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.400+0000 7f23c5ffb640 1 -- 172.21.15.7:0/1271342328 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f23bc0b91e0 con 0x7f23c8100650 2024-03-23T13:17:38.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:38 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] up:boot 2024-03-23T13:17:38.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:38 smithi130 ceph-mon[36225]: fsmap 12 up:standby 2024-03-23T13:17:38.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:38 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T13:17:38.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:38 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/4276854815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:38.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:38 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:17:38.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:38 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2805703320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:38.444 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:38 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1802593365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:17:38.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:38 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] up:boot 2024-03-23T13:17:38.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:38 smithi155 ceph-mon[36157]: fsmap 12 up:standby 2024-03-23T13:17:38.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:38 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T13:17:38.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:38 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/4276854815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:38.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:38 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:17:38.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:38 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2805703320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:38.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:38 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1802593365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:17:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:38.584+0000 7f23ceadf640 1 -- 172.21.15.7:0/1271342328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f23c802d280 con 0x7f23c8100650 2024-03-23T13:17:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.209+0000 7f23c5ffb640 1 -- 172.21.15.7:0/1271342328 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f23bc0bc8a0 con 0x7f23c8100650 2024-03-23T13:17:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-03-23T13:17:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 -- 172.21.15.7:0/1271342328 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f23b80a20c0 msgr2=0x7f23b80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f23b80a20c0 0x7f23b80a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005fd0 tx=0x7f23b4005850 comp rx=0 tx=0).stop 2024-03-23T13:17:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 -- 172.21.15.7:0/1271342328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c8100650 msgr2=0x7f23c8196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c8100650 0x7f23c8196790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23bc005ab0 tx=0x7f23bc004270 comp rx=0 tx=0).stop 2024-03-23T13:17:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 -- 172.21.15.7:0/1271342328 shutdown_connections 2024-03-23T13:17:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f23b80a20c0 0x7f23b80a4580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23c8102200 0x7f23c819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c8101840 0x7f23c8196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 --2- 172.21.15.7:0/1271342328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c8100650 0x7f23c8196790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 -- 172.21.15.7:0/1271342328 >> 172.21.15.7:0/1271342328 conn(0x7f23c80fbbc0 msgr2=0x7f23c8104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 -- 172.21.15.7:0/1271342328 shutdown_connections 2024-03-23T13:17:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.211+0000 7f23ceadf640 1 -- 172.21.15.7:0/1271342328 wait complete. 2024-03-23T13:17:39.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.334+0000 7f754a91e640 1 -- 172.21.15.7:0/2703911155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75441021d0 msgr2=0x7f7544108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.334+0000 7f754a91e640 1 --2- 172.21.15.7:0/2703911155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75441021d0 0x7f7544108a60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f753c009ca0 tx=0x7f753c02c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.334+0000 7f754a91e640 1 -- 172.21.15.7:0/2703911155 shutdown_connections 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.334+0000 7f754a91e640 1 --2- 172.21.15.7:0/2703911155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75441021d0 0x7f7544108a60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.334+0000 7f754a91e640 1 --2- 172.21.15.7:0/2703911155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7544101810 0x7f7544101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.334+0000 7f754a91e640 1 --2- 172.21.15.7:0/2703911155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7544100620 0x7f7544100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.334+0000 7f754a91e640 1 -- 172.21.15.7:0/2703911155 >> 172.21.15.7:0/2703911155 conn(0x7f75440fbbb0 msgr2=0x7f75440fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.334+0000 7f754a91e640 1 -- 172.21.15.7:0/2703911155 shutdown_connections 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.334+0000 7f754a91e640 1 -- 172.21.15.7:0/2703911155 wait complete. 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f754a91e640 1 Processor -- start 2024-03-23T13:17:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f754a91e640 1 -- start start 2024-03-23T13:17:39.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f754a91e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7544100620 0x7f7544197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:39.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f754a91e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7544101810 0x7f7544197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:39.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f754a91e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75441021d0 0x7f754419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f754a91e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f754419f200 con 0x7f7544101810 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f754a91e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f754419f370 con 0x7f75441021d0 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f754a91e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f754419f530 con 0x7f7544100620 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f75437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7544101810 0x7f7544197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f75437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7544101810 0x7f7544197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36034/0 (socket says 172.21.15.7:36034) 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f75437fe640 1 -- 172.21.15.7:0/796360541 learned_addr learned my addr 172.21.15.7:0/796360541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f7543fff640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7544100620 0x7f7544197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f7548ebb640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75441021d0 0x7f754419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f75437fe640 1 -- 172.21.15.7:0/796360541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7544100620 msgr2=0x7f7544197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f75437fe640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7544100620 0x7f7544197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f75437fe640 1 -- 172.21.15.7:0/796360541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75441021d0 msgr2=0x7f754419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f75437fe640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75441021d0 0x7f754419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:39.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f75437fe640 1 -- 172.21.15.7:0/796360541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c009900 con 0x7f7544101810 2024-03-23T13:17:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.335+0000 7f7548ebb640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75441021d0 0x7f754419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:17:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.336+0000 7f75437fe640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7544101810 0x7f7544197c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f753400e9f0 tx=0x7f753400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.336+0000 7f75417fa640 1 -- 172.21.15.7:0/796360541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534009720 con 0x7f7544101810 2024-03-23T13:17:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.336+0000 7f75417fa640 1 -- 172.21.15.7:0/796360541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f753400cd80 con 0x7f7544101810 2024-03-23T13:17:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.336+0000 7f754a91e640 1 -- 172.21.15.7:0/796360541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754419f810 con 0x7f7544101810 2024-03-23T13:17:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.336+0000 7f75417fa640 1 -- 172.21.15.7:0/796360541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534010690 con 0x7f7544101810 2024-03-23T13:17:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.336+0000 7f754a91e640 1 -- 172.21.15.7:0/796360541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754419fcd0 con 0x7f7544101810 2024-03-23T13:17:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.337+0000 7f754a91e640 1 -- 172.21.15.7:0/796360541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754410c030 con 0x7f7544101810 2024-03-23T13:17:39.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.338+0000 7f75417fa640 1 -- 172.21.15.7:0/796360541 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f75340046c0 con 0x7f7544101810 2024-03-23T13:17:39.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.339+0000 7f75417fa640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f75300a20c0 0x7f75300a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:39.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.339+0000 7f7543fff640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f75300a20c0 0x7f75300a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:39.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.339+0000 7f75417fa640 1 -- 172.21.15.7:0/796360541 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f7534014070 con 0x7f7544101810 2024-03-23T13:17:39.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.339+0000 7f7543fff640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f75300a20c0 0x7f75300a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7544101670 tx=0x7f752c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:39.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.342+0000 7f75417fa640 1 -- 172.21.15.7:0/796360541 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f75340972b0 con 0x7f7544101810 2024-03-23T13:17:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:39.530+0000 7f754a91e640 1 -- 172.21.15.7:0/796360541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f754410c1c0 con 0x7f7544101810 2024-03-23T13:17:39.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:39 smithi155 ceph-mon[36157]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:39.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:39 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3037485274' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T13:17:39.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:39 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1271342328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T13:17:39.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:39 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:39 smithi007 ceph-mon[32184]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:39.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:39 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3037485274' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T13:17:39.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:39 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1271342328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T13:17:39.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:39 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:39 smithi130 ceph-mon[36225]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:39.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:39 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3037485274' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T13:17:39.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:39 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1271342328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T13:17:39.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:39.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:39 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:40.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.211+0000 7f75417fa640 1 -- 172.21.15.7:0/796360541 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f753409a970 con 0x7f7544101810 2024-03-23T13:17:40.212 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-03-23T13:17:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.214+0000 7f754a91e640 1 -- 172.21.15.7:0/796360541 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f75300a20c0 msgr2=0x7f75300a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.214+0000 7f754a91e640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f75300a20c0 0x7f75300a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7544101670 tx=0x7f752c005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.214+0000 7f754a91e640 1 -- 172.21.15.7:0/796360541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7544101810 msgr2=0x7f7544197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.214+0000 7f754a91e640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7544101810 0x7f7544197c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f753400e9f0 tx=0x7f753400edf0 comp rx=0 tx=0).stop 2024-03-23T13:17:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.214+0000 7f754a91e640 1 -- 172.21.15.7:0/796360541 shutdown_connections 2024-03-23T13:17:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.214+0000 7f754a91e640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f75300a20c0 0x7f75300a4580 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.214+0000 7f754a91e640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75441021d0 0x7f754419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.214+0000 7f754a91e640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7544101810 0x7f7544197c50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.214+0000 7f754a91e640 1 --2- 172.21.15.7:0/796360541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7544100620 0x7f7544197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.214+0000 7f754a91e640 1 -- 172.21.15.7:0/796360541 >> 172.21.15.7:0/796360541 conn(0x7f75440fbbb0 msgr2=0x7f7544069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:40.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.217+0000 7f754a91e640 1 -- 172.21.15.7:0/796360541 shutdown_connections 2024-03-23T13:17:40.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.217+0000 7f754a91e640 1 -- 172.21.15.7:0/796360541 wait complete. 2024-03-23T13:17:40.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T13:17:40.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.360+0000 7f80b4e84640 1 -- 172.21.15.7:0/2729015122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80b0107470 msgr2=0x7f80b01098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.360+0000 7f80b4e84640 1 --2- 172.21.15.7:0/2729015122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80b0107470 0x7f80b01098c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009900 tx=0x7f80a802bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.360+0000 7f80b4e84640 1 -- 172.21.15.7:0/2729015122 shutdown_connections 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.360+0000 7f80b4e84640 1 --2- 172.21.15.7:0/2729015122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80b0107470 0x7f80b01098c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.360+0000 7f80b4e84640 1 --2- 172.21.15.7:0/2729015122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b0104b40 0x7f80b0106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.360+0000 7f80b4e84640 1 --2- 172.21.15.7:0/2729015122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80b0102210 0x7f80b0104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.360+0000 7f80b4e84640 1 -- 172.21.15.7:0/2729015122 >> 172.21.15.7:0/2729015122 conn(0x7f80b00fbc40 msgr2=0x7f80b00fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.360+0000 7f80b4e84640 1 -- 172.21.15.7:0/2729015122 shutdown_connections 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.360+0000 7f80b4e84640 1 -- 172.21.15.7:0/2729015122 wait complete. 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80b4e84640 1 Processor -- start 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80b4e84640 1 -- start start 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80b4e84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80b0102210 0x7f80b0197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80b4e84640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b0104b40 0x7f80b0197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80b4e84640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80b0107470 0x7f80b019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80b4e84640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80b019f250 con 0x7f80b0102210 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80b4e84640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f80b019f3c0 con 0x7f80b0107470 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80b4e84640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80b019f580 con 0x7f80b0104b40 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80ae59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80b0102210 0x7f80b0197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80ae59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80b0102210 0x7f80b0197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36046/0 (socket says 172.21.15.7:36046) 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80ae59c640 1 -- 172.21.15.7:0/2311340036 learned_addr learned my addr 172.21.15.7:0/2311340036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80aed9d640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80b0107470 0x7f80b019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.361+0000 7f80add9b640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b0104b40 0x7f80b0197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f80ae59c640 1 -- 172.21.15.7:0/2311340036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b0104b40 msgr2=0x7f80b0197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f80ae59c640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b0104b40 0x7f80b0197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f80ae59c640 1 -- 172.21.15.7:0/2311340036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80b0107470 msgr2=0x7f80b019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f80ae59c640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80b0107470 0x7f80b019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f80ae59c640 1 -- 172.21.15.7:0/2311340036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a4009490 con 0x7f80b0102210 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f80aed9d640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80b0107470 0x7f80b019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:17:40.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f80ae59c640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80b0102210 0x7f80b0197760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f80a000ebc0 tx=0x7f80a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:40.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f809f7fe640 1 -- 172.21.15.7:0/2311340036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a00099b0 con 0x7f80b0102210 2024-03-23T13:17:40.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f809f7fe640 1 -- 172.21.15.7:0/2311340036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a000eec0 con 0x7f80b0102210 2024-03-23T13:17:40.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f80b4e84640 1 -- 172.21.15.7:0/2311340036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a8009560 con 0x7f80b0102210 2024-03-23T13:17:40.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f809f7fe640 1 -- 172.21.15.7:0/2311340036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0018b60 con 0x7f80b0102210 2024-03-23T13:17:40.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.362+0000 7f80b4e84640 1 -- 172.21.15.7:0/2311340036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b019fbe0 con 0x7f80b0102210 2024-03-23T13:17:40.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.363+0000 7f80b4e84640 1 -- 172.21.15.7:0/2311340036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b0068950 con 0x7f80b0102210 2024-03-23T13:17:40.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.364+0000 7f809f7fe640 1 -- 172.21.15.7:0/2311340036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f80a0018cc0 con 0x7f80b0102210 2024-03-23T13:17:40.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.365+0000 7f809f7fe640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f80980a20c0 0x7f80980a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:40.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.365+0000 7f80add9b640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f80980a20c0 0x7f80980a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:40.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.365+0000 7f809f7fe640 1 -- 172.21.15.7:0/2311340036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f80a0014070 con 0x7f80b0102210 2024-03-23T13:17:40.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.366+0000 7f80add9b640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f80980a20c0 0x7f80980a4580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80b0198c70 tx=0x7f80a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:40.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.370+0000 7f809f7fe640 1 -- 172.21.15.7:0/2311340036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f80a0053280 con 0x7f80b0102210 2024-03-23T13:17:40.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:40 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1271342328' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T13:17:40.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:40 smithi155 ceph-mon[36157]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T13:17:40.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:40 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:40.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:40 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:17:40.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:40 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:40.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:40 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/796360541' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T13:17:40.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:40 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:17:40.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:40 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1271342328' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T13:17:40.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:40 smithi007 ceph-mon[32184]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T13:17:40.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:40 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:40.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:40 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:17:40.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:40 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:40.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:40 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/796360541' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T13:17:40.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:40 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:17:40.588 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:40 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1271342328' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T13:17:40.589 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:40 smithi130 ceph-mon[36225]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T13:17:40.589 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:40 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:40.589 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:40 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:17:40.589 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:40 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:40.589 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:40 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/796360541' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T13:17:40.589 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:40 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:17:40.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:40.594+0000 7f80b4e84640 1 -- 172.21.15.7:0/2311340036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f80b00fd730 con 0x7f80b0102210 2024-03-23T13:17:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.520+0000 7f809f7fe640 1 -- 172.21.15.7:0/2311340036 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f80a009ae50 con 0x7f80b0102210 2024-03-23T13:17:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-03-23T13:17:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 -- 172.21.15.7:0/2311340036 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f80980a20c0 msgr2=0x7f80980a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f80980a20c0 0x7f80980a4580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80b0198c70 tx=0x7f80a4005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 -- 172.21.15.7:0/2311340036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80b0102210 msgr2=0x7f80b0197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80b0102210 0x7f80b0197760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f80a000ebc0 tx=0x7f80a000c6a0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 -- 172.21.15.7:0/2311340036 shutdown_connections 2024-03-23T13:17:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f80980a20c0 0x7f80980a4580 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80b0107470 0x7f80b019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80b0104b40 0x7f80b0197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 --2- 172.21.15.7:0/2311340036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80b0102210 0x7f80b0197760 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 -- 172.21.15.7:0/2311340036 >> 172.21.15.7:0/2311340036 conn(0x7f80b00fbc40 msgr2=0x7f80b0102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 -- 172.21.15.7:0/2311340036 shutdown_connections 2024-03-23T13:17:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.522+0000 7f80b4e84640 1 -- 172.21.15.7:0/2311340036 wait complete. 2024-03-23T13:17:41.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:41 smithi007 ceph-mon[32184]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:41.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:41 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/796360541' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T13:17:41.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:41 smithi007 ceph-mon[32184]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T13:17:41.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:41 smithi007 ceph-mon[32184]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-23T13:17:41.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:41 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2311340036' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T13:17:41.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:41 smithi007 ceph-mon[32184]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T13:17:41.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:17:41.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:41 smithi155 ceph-mon[36157]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:41.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:41 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/796360541' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T13:17:41.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:41 smithi155 ceph-mon[36157]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T13:17:41.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:41 smithi155 ceph-mon[36157]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-23T13:17:41.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:41 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2311340036' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T13:17:41.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:41 smithi155 ceph-mon[36157]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T13:17:41.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 -- 172.21.15.7:0/3610511523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b01021e0 msgr2=0x7f26b0108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:41.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 --2- 172.21.15.7:0/3610511523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b01021e0 0x7f26b0108a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009ca0 tx=0x7f26a802c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 -- 172.21.15.7:0/3610511523 shutdown_connections 2024-03-23T13:17:41.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 --2- 172.21.15.7:0/3610511523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b01021e0 0x7f26b0108a70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 --2- 172.21.15.7:0/3610511523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26b0101820 0x7f26b0101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 --2- 172.21.15.7:0/3610511523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b0100630 0x7f26b0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 -- 172.21.15.7:0/3610511523 >> 172.21.15.7:0/3610511523 conn(0x7f26b00fbba0 msgr2=0x7f26b00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 -- 172.21.15.7:0/3610511523 shutdown_connections 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 -- 172.21.15.7:0/3610511523 wait complete. 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 Processor -- start 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 -- start start 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b0100630 0x7f26b0069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b0101820 0x7f26b019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26b01021e0 0x7f26b019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26b019f240 con 0x7f26b0100630 2024-03-23T13:17:41.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f26b019f3b0 con 0x7f26b01021e0 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.640+0000 7f26b8519640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26b019f570 con 0x7f26b0101820 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b62b5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b0100630 0x7f26b0069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b62b5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b0100630 0x7f26b0069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36060/0 (socket says 172.21.15.7:36060) 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b62b5640 1 -- 172.21.15.7:0/3061970881 learned_addr learned my addr 172.21.15.7:0/3061970881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b5ab4640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b0101820 0x7f26b019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b62b5640 1 -- 172.21.15.7:0/3061970881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b0101820 msgr2=0x7f26b019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b62b5640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b0101820 0x7f26b019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b6ab6640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26b01021e0 0x7f26b019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b62b5640 1 -- 172.21.15.7:0/3061970881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26b01021e0 msgr2=0x7f26b019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b62b5640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26b01021e0 0x7f26b019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b62b5640 1 -- 172.21.15.7:0/3061970881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a8009900 con 0x7f26b0100630 2024-03-23T13:17:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b62b5640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b0100630 0x7f26b0069440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00d950 tx=0x7f26ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:41.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26a77fe640 1 -- 172.21.15.7:0/3061970881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac009c30 con 0x7f26b0100630 2024-03-23T13:17:41.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b8519640 1 -- 172.21.15.7:0/3061970881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b019f850 con 0x7f26b0100630 2024-03-23T13:17:41.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26a77fe640 1 -- 172.21.15.7:0/3061970881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac009d90 con 0x7f26b0100630 2024-03-23T13:17:41.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26a77fe640 1 -- 172.21.15.7:0/3061970881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00f720 con 0x7f26b0100630 2024-03-23T13:17:41.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.641+0000 7f26b8519640 1 -- 172.21.15.7:0/3061970881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b019fd10 con 0x7f26b0100630 2024-03-23T13:17:41.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.642+0000 7f26b8519640 1 -- 172.21.15.7:0/3061970881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b0107950 con 0x7f26b0100630 2024-03-23T13:17:41.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.643+0000 7f26a77fe640 1 -- 172.21.15.7:0/3061970881 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f26ac010a60 con 0x7f26b0100630 2024-03-23T13:17:41.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.644+0000 7f26a77fe640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f269c0a20c0 0x7f269c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:41.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.644+0000 7f26b5ab4640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f269c0a20c0 0x7f269c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:41.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.644+0000 7f26a77fe640 1 -- 172.21.15.7:0/3061970881 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f26ac0cd9a0 con 0x7f26b0100630 2024-03-23T13:17:41.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.644+0000 7f26b5ab4640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f269c0a20c0 0x7f269c0a4580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f26b00691b0 tx=0x7f26a000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:41.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.649+0000 7f26a77fe640 1 -- 172.21.15.7:0/3061970881 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f26ac096a20 con 0x7f26b0100630 2024-03-23T13:17:41.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:41 smithi130 ceph-mon[36225]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T13:17:41.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:41 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/796360541' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T13:17:41.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:41 smithi130 ceph-mon[36225]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T13:17:41.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:41 smithi130 ceph-mon[36225]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-23T13:17:41.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:41 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2311340036' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T13:17:41.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:41 smithi130 ceph-mon[36225]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T13:17:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.831+0000 7f26b8519640 1 -- 172.21.15.7:0/3061970881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26b002d1d0 con 0x7f26b0100630 2024-03-23T13:17:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.833+0000 7f26a77fe640 1 -- 172.21.15.7:0/3061970881 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22137 (secure 0 0 0) 0x7f26ac09a0e0 con 0x7f26b0100630 2024-03-23T13:17:41.832 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:41.832 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:17:40.596874+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:17:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.834+0000 7f26b8519640 1 -- 172.21.15.7:0/3061970881 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f269c0a20c0 msgr2=0x7f269c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f269c0a20c0 0x7f269c0a4580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f26b00691b0 tx=0x7f26a000a2d0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 -- 172.21.15.7:0/3061970881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b0100630 msgr2=0x7f26b0069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b0100630 0x7f26b0069440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00d950 tx=0x7f26ac00de10 comp rx=0 tx=0).stop 2024-03-23T13:17:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 -- 172.21.15.7:0/3061970881 shutdown_connections 2024-03-23T13:17:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f269c0a20c0 0x7f269c0a4580 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26b01021e0 0x7f26b019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b0101820 0x7f26b019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 --2- 172.21.15.7:0/3061970881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26b0100630 0x7f26b0069440 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 -- 172.21.15.7:0/3061970881 >> 172.21.15.7:0/3061970881 conn(0x7f26b00fbba0 msgr2=0x7f26b0104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 -- 172.21.15.7:0/3061970881 shutdown_connections 2024-03-23T13:17:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.835+0000 7f26b8519640 1 -- 172.21.15.7:0/3061970881 wait complete. 2024-03-23T13:17:41.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:17:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.949+0000 7f2180064640 1 -- 172.21.15.7:0/2959791266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178101730 msgr2=0x7f2178101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.949+0000 7f2180064640 1 --2- 172.21.15.7:0/2959791266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178101730 0x7f2178101b50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2174009ca0 tx=0x7f217402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 -- 172.21.15.7:0/2959791266 shutdown_connections 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 --2- 172.21.15.7:0/2959791266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21781032e0 0x7f2178109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 --2- 172.21.15.7:0/2959791266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2178102920 0x7f2178102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 --2- 172.21.15.7:0/2959791266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178101730 0x7f2178101b50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 -- 172.21.15.7:0/2959791266 >> 172.21.15.7:0/2959791266 conn(0x7f21780fccc0 msgr2=0x7f21780ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 -- 172.21.15.7:0/2959791266 shutdown_connections 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 -- 172.21.15.7:0/2959791266 wait complete. 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 Processor -- start 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 -- start start 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2178101730 0x7f2178197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2178102920 0x7f2178197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21781032e0 0x7f217819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f217819f2f0 con 0x7f21781032e0 2024-03-23T13:17:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f217819f460 con 0x7f2178101730 2024-03-23T13:17:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.950+0000 7f2180064640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f217819f620 con 0x7f2178102920 2024-03-23T13:17:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217e601640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21781032e0 0x7f217819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217e601640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21781032e0 0x7f217819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36080/0 (socket says 172.21.15.7:36080) 2024-03-23T13:17:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217e601640 1 -- 172.21.15.7:0/1807956742 learned_addr learned my addr 172.21.15.7:0/1807956742 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217de00640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2178101730 0x7f2178197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217d5ff640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2178102920 0x7f2178197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217e601640 1 -- 172.21.15.7:0/1807956742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2178102920 msgr2=0x7f2178197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217e601640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2178102920 0x7f2178197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217e601640 1 -- 172.21.15.7:0/1807956742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2178101730 msgr2=0x7f2178197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217e601640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2178101730 0x7f2178197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217e601640 1 -- 172.21.15.7:0/1807956742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2174009900 con 0x7f21781032e0 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f217e601640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21781032e0 0x7f217819edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f217000c970 tx=0x7f217000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f216effd640 1 -- 172.21.15.7:0/1807956742 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170007bb0 con 0x7f21781032e0 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f2180064640 1 -- 172.21.15.7:0/1807956742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217819f900 con 0x7f21781032e0 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f216effd640 1 -- 172.21.15.7:0/1807956742 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2170007d10 con 0x7f21781032e0 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f216effd640 1 -- 172.21.15.7:0/1807956742 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217000f450 con 0x7f21781032e0 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.951+0000 7f2180064640 1 -- 172.21.15.7:0/1807956742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217819fe10 con 0x7f21781032e0 2024-03-23T13:17:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.952+0000 7f2180064640 1 -- 172.21.15.7:0/1807956742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2178108a70 con 0x7f21781032e0 2024-03-23T13:17:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.953+0000 7f216effd640 1 -- 172.21.15.7:0/1807956742 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f2170004ee0 con 0x7f21781032e0 2024-03-23T13:17:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.954+0000 7f216effd640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f21640a20c0 0x7f21640a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.954+0000 7f217de00640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f21640a20c0 0x7f21640a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.954+0000 7f216effd640 1 -- 172.21.15.7:0/1807956742 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f21700cd720 con 0x7f21781032e0 2024-03-23T13:17:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.954+0000 7f217de00640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f21640a20c0 0x7f21640a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f217402c700 tx=0x7f21740326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:41.959+0000 7f216effd640 1 -- 172.21.15.7:0/1807956742 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f21700968f0 con 0x7f21781032e0 2024-03-23T13:17:42.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.142+0000 7f2180064640 1 -- 172.21.15.7:0/1807956742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f217810d320 con 0x7f21781032e0 2024-03-23T13:17:42.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.143+0000 7f216effd640 1 -- 172.21.15.7:0/1807956742 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22137 (secure 0 0 0) 0x7f2170099fb0 con 0x7f21781032e0 2024-03-23T13:17:42.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:42.354 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:17:40.596874+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:17:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 -- 172.21.15.7:0/1807956742 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f21640a20c0 msgr2=0x7f21640a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f21640a20c0 0x7f21640a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f217402c700 tx=0x7f21740326b0 comp rx=0 tx=0).stop 2024-03-23T13:17:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 -- 172.21.15.7:0/1807956742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21781032e0 msgr2=0x7f217819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21781032e0 0x7f217819edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f217000c970 tx=0x7f217000ce30 comp rx=0 tx=0).stop 2024-03-23T13:17:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 -- 172.21.15.7:0/1807956742 shutdown_connections 2024-03-23T13:17:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f21640a20c0 0x7f21640a4580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21781032e0 0x7f217819edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2178102920 0x7f2178197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 --2- 172.21.15.7:0/1807956742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2178101730 0x7f2178197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 -- 172.21.15.7:0/1807956742 >> 172.21.15.7:0/1807956742 conn(0x7f21780fccc0 msgr2=0x7f2178105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 -- 172.21.15.7:0/1807956742 shutdown_connections 2024-03-23T13:17:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.145+0000 7f2180064640 1 -- 172.21.15.7:0/1807956742 wait complete. 2024-03-23T13:17:42.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T13:17:42.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.464+0000 7f27a754f640 1 -- 172.21.15.7:0/3318546412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a01021e0 msgr2=0x7f27a0108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.464+0000 7f27a754f640 1 --2- 172.21.15.7:0/3318546412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a01021e0 0x7f27a0108a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2798009900 tx=0x7f279802bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.464+0000 7f27a754f640 1 -- 172.21.15.7:0/3318546412 shutdown_connections 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.464+0000 7f27a754f640 1 --2- 172.21.15.7:0/3318546412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a01021e0 0x7f27a0108a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.464+0000 7f27a754f640 1 --2- 172.21.15.7:0/3318546412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27a0101820 0x7f27a0101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.464+0000 7f27a754f640 1 --2- 172.21.15.7:0/3318546412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27a0100630 0x7f27a0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.464+0000 7f27a754f640 1 -- 172.21.15.7:0/3318546412 >> 172.21.15.7:0/3318546412 conn(0x7f27a00fbbc0 msgr2=0x7f27a00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.464+0000 7f27a754f640 1 -- 172.21.15.7:0/3318546412 shutdown_connections 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.464+0000 7f27a754f640 1 -- 172.21.15.7:0/3318546412 wait complete. 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a754f640 1 Processor -- start 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a754f640 1 -- start start 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a754f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27a0100630 0x7f27a0197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a754f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27a0101820 0x7f27a0197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a754f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a01021e0 0x7f27a019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a754f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27a019f210 con 0x7f27a01021e0 2024-03-23T13:17:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a754f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f27a019f380 con 0x7f27a0100630 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a754f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27a019f540 con 0x7f27a0101820 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a5aec640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a01021e0 0x7f27a019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a5aec640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a01021e0 0x7f27a019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36098/0 (socket says 172.21.15.7:36098) 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a5aec640 1 -- 172.21.15.7:0/3846503166 learned_addr learned my addr 172.21.15.7:0/3846503166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a52eb640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27a0100630 0x7f27a0197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a4aea640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27a0101820 0x7f27a0197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a5aec640 1 -- 172.21.15.7:0/3846503166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27a0101820 msgr2=0x7f27a0197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a5aec640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27a0101820 0x7f27a0197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a5aec640 1 -- 172.21.15.7:0/3846503166 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27a0100630 msgr2=0x7f27a0197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a5aec640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27a0100630 0x7f27a0197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a5aec640 1 -- 172.21.15.7:0/3846503166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2798009560 con 0x7f27a01021e0 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.465+0000 7f27a52eb640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27a0100630 0x7f27a0197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.466+0000 7f27a5aec640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a01021e0 0x7f27a019ecd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2798009230 tx=0x7f2798004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.466+0000 7f279e7fc640 1 -- 172.21.15.7:0/3846503166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798004df0 con 0x7f27a01021e0 2024-03-23T13:17:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.466+0000 7f279e7fc640 1 -- 172.21.15.7:0/3846503166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2798005430 con 0x7f27a01021e0 2024-03-23T13:17:42.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.466+0000 7f279e7fc640 1 -- 172.21.15.7:0/3846503166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798003de0 con 0x7f27a01021e0 2024-03-23T13:17:42.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.466+0000 7f27a754f640 1 -- 172.21.15.7:0/3846503166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a019f7c0 con 0x7f27a01021e0 2024-03-23T13:17:42.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.466+0000 7f27a754f640 1 -- 172.21.15.7:0/3846503166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a019fc80 con 0x7f27a01021e0 2024-03-23T13:17:42.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.467+0000 7f27a754f640 1 -- 172.21.15.7:0/3846503166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a010c040 con 0x7f27a01021e0 2024-03-23T13:17:42.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.468+0000 7f279e7fc640 1 -- 172.21.15.7:0/3846503166 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f27980055a0 con 0x7f27a01021e0 2024-03-23T13:17:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.468+0000 7f279e7fc640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f27880a2140 0x7f27880a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.469+0000 7f27a52eb640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f27880a2140 0x7f27880a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.469+0000 7f279e7fc640 1 -- 172.21.15.7:0/3846503166 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f27980f0140 con 0x7f27a01021e0 2024-03-23T13:17:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.469+0000 7f27a52eb640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f27880a2140 0x7f27880a4600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f27a0101680 tx=0x7f2790007870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:42.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.473+0000 7f279e7fc640 1 -- 172.21.15.7:0/3846503166 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f27980b9260 con 0x7f27a01021e0 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2311340036' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3061970881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:42.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:42 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1807956742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2311340036' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T13:17:42.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3061970881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:42.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:42 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1807956742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2311340036' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T13:17:42.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T13:17:42.695 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.695 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T13:17:42.695 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3061970881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:42.695 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:17:42.695 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:42 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1807956742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:42.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:42.698+0000 7f27a754f640 1 -- 172.21.15.7:0/3846503166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f27a019fea0 con 0x7f27a01021e0 2024-03-23T13:17:43.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.533+0000 7f279e7fc640 1 -- 172.21.15.7:0/3846503166 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f27980bc920 con 0x7f27a01021e0 2024-03-23T13:17:43.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 -- 172.21.15.7:0/3846503166 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f27880a2140 msgr2=0x7f27880a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:43.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f27880a2140 0x7f27880a4600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f27a0101680 tx=0x7f2790007870 comp rx=0 tx=0).stop 2024-03-23T13:17:43.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 -- 172.21.15.7:0/3846503166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a01021e0 msgr2=0x7f27a019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:43.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a01021e0 0x7f27a019ecd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2798009230 tx=0x7f2798004270 comp rx=0 tx=0).stop 2024-03-23T13:17:43.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 -- 172.21.15.7:0/3846503166 shutdown_connections 2024-03-23T13:17:43.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f27880a2140 0x7f27880a4600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:43.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a01021e0 0x7f27a019ecd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:43.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27a0101820 0x7f27a0197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:43.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 --2- 172.21.15.7:0/3846503166 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27a0100630 0x7f27a0197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:43.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 -- 172.21.15.7:0/3846503166 >> 172.21.15.7:0/3846503166 conn(0x7f27a00fbbc0 msgr2=0x7f27a00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:43.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 -- 172.21.15.7:0/3846503166 shutdown_connections 2024-03-23T13:17:43.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.536+0000 7f27a754f640 1 -- 172.21.15.7:0/3846503166 wait complete. 2024-03-23T13:17:43.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:43 smithi155 ceph-mon[36157]: pgmap v284: 97 pgs: 18 creating+peering, 52 unknown, 27 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-03-23T13:17:43.537 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:43 smithi155 ceph-mon[36157]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T13:17:43.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:43 smithi155 ceph-mon[36157]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-23T13:17:43.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:43 smithi155 ceph-mon[36157]: Cluster is now healthy 2024-03-23T13:17:43.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:43 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] up:active 2024-03-23T13:17:43.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:43 smithi155 ceph-mon[36157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:43.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:43 smithi155 ceph-mon[36157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:43.538 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:43 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3846503166' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T13:17:43.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:43 smithi007 ceph-mon[32184]: pgmap v284: 97 pgs: 18 creating+peering, 52 unknown, 27 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-03-23T13:17:43.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:43 smithi007 ceph-mon[32184]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T13:17:43.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:43 smithi007 ceph-mon[32184]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-23T13:17:43.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:43 smithi007 ceph-mon[32184]: Cluster is now healthy 2024-03-23T13:17:43.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:43 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] up:active 2024-03-23T13:17:43.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:43 smithi007 ceph-mon[32184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:43.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:43 smithi007 ceph-mon[32184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:43.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:43 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3846503166' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T13:17:43.550 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-23T13:17:43.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 -- 172.21.15.7:0/2386541130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d8101740 msgr2=0x7f72d8101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 --2- 172.21.15.7:0/2386541130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d8101740 0x7f72d8101b60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009900 tx=0x7f72d402bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 -- 172.21.15.7:0/2386541130 shutdown_connections 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 --2- 172.21.15.7:0/2386541130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72d81032f0 0x7f72d8109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 --2- 172.21.15.7:0/2386541130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d8102930 0x7f72d8102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 --2- 172.21.15.7:0/2386541130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d8101740 0x7f72d8101b60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 -- 172.21.15.7:0/2386541130 >> 172.21.15.7:0/2386541130 conn(0x7f72d80fccb0 msgr2=0x7f72d80ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 -- 172.21.15.7:0/2386541130 shutdown_connections 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 -- 172.21.15.7:0/2386541130 wait complete. 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 Processor -- start 2024-03-23T13:17:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 -- start start 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72d8101740 0x7f72d8197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d8102930 0x7f72d8197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d81032f0 0x7f72d819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.663+0000 7f72dfbdd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72d819f340 con 0x7f72d81032f0 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72dfbdd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f72d819f4b0 con 0x7f72d8101740 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72dfbdd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72d819f670 con 0x7f72d8102930 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72de17a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d81032f0 0x7f72d819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72de17a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d81032f0 0x7f72d819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36118/0 (socket says 172.21.15.7:36118) 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72de17a640 1 -- 172.21.15.7:0/4148330124 learned_addr learned my addr 172.21.15.7:0/4148330124 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72dd979640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72d8101740 0x7f72d8197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72dd178640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d8102930 0x7f72d8197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:43.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72de17a640 1 -- 172.21.15.7:0/4148330124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d8102930 msgr2=0x7f72d8197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72de17a640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d8102930 0x7f72d8197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72de17a640 1 -- 172.21.15.7:0/4148330124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72d8101740 msgr2=0x7f72d8197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72de17a640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72d8101740 0x7f72d8197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72de17a640 1 -- 172.21.15.7:0/4148330124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d4009560 con 0x7f72d81032f0 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72dd178640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d8102930 0x7f72d8197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72de17a640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d81032f0 0x7f72d819ee00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f72d000eb10 tx=0x7f72d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72ceffd640 1 -- 172.21.15.7:0/4148330124 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0004b20 con 0x7f72d81032f0 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72ceffd640 1 -- 172.21.15.7:0/4148330124 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d0004c80 con 0x7f72d81032f0 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72dfbdd640 1 -- 172.21.15.7:0/4148330124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d819f950 con 0x7f72d81032f0 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72ceffd640 1 -- 172.21.15.7:0/4148330124 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0010650 con 0x7f72d81032f0 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.664+0000 7f72dfbdd640 1 -- 172.21.15.7:0/4148330124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d819fee0 con 0x7f72d81032f0 2024-03-23T13:17:43.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.665+0000 7f72dfbdd640 1 -- 172.21.15.7:0/4148330124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d8108ae0 con 0x7f72d81032f0 2024-03-23T13:17:43.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.667+0000 7f72ceffd640 1 -- 172.21.15.7:0/4148330124 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f72d00107f0 con 0x7f72d81032f0 2024-03-23T13:17:43.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.668+0000 7f72ceffd640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72c40a2140 0x7f72c40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:43.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.668+0000 7f72dd979640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72c40a2140 0x7f72c40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:43.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.668+0000 7f72ceffd640 1 -- 172.21.15.7:0/4148330124 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f72d0014070 con 0x7f72d81032f0 2024-03-23T13:17:43.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.668+0000 7f72dd979640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72c40a2140 0x7f72c40a4600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72d4005ab0 tx=0x7f72d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:43.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.671+0000 7f72ceffd640 1 -- 172.21.15.7:0/4148330124 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f72d00973b0 con 0x7f72d81032f0 2024-03-23T13:17:43.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:43 smithi130 ceph-mon[36225]: pgmap v284: 97 pgs: 18 creating+peering, 52 unknown, 27 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-03-23T13:17:43.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:43 smithi130 ceph-mon[36225]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T13:17:43.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:43 smithi130 ceph-mon[36225]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-23T13:17:43.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:43 smithi130 ceph-mon[36225]: Cluster is now healthy 2024-03-23T13:17:43.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:43 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] up:active 2024-03-23T13:17:43.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:43 smithi130 ceph-mon[36225]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:43.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:43 smithi130 ceph-mon[36225]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:43.694 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:43 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3846503166' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T13:17:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:43.892+0000 7f72dfbdd640 1 -- 172.21.15.7:0/4148330124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f72d802d1d0 con 0x7f72d81032f0 2024-03-23T13:17:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.539+0000 7f72ceffd640 1 -- 172.21.15.7:0/4148330124 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f72d009aa70 con 0x7f72d81032f0 2024-03-23T13:17:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 -- 172.21.15.7:0/4148330124 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72c40a2140 msgr2=0x7f72c40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72c40a2140 0x7f72c40a4600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72d4005ab0 tx=0x7f72d4005fd0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 -- 172.21.15.7:0/4148330124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d81032f0 msgr2=0x7f72d819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d81032f0 0x7f72d819ee00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f72d000eb10 tx=0x7f72d000c2d0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 -- 172.21.15.7:0/4148330124 shutdown_connections 2024-03-23T13:17:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72c40a2140 0x7f72c40a4600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d81032f0 0x7f72d819ee00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d8102930 0x7f72d8197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 --2- 172.21.15.7:0/4148330124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72d8101740 0x7f72d8197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 -- 172.21.15.7:0/4148330124 >> 172.21.15.7:0/4148330124 conn(0x7f72d80fccb0 msgr2=0x7f72d8105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 -- 172.21.15.7:0/4148330124 shutdown_connections 2024-03-23T13:17:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.542+0000 7f72dfbdd640 1 -- 172.21.15.7:0/4148330124 wait complete. 2024-03-23T13:17:44.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-23T13:17:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.669+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/2115889690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c08102910 msgr2=0x7f7c08102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.669+0000 7f7c0ef3e640 1 --2- 172.21.15.7:0/2115889690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c08102910 0x7f7c08102d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009ca0 tx=0x7f7bf802c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.669+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/2115889690 shutdown_connections 2024-03-23T13:17:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.669+0000 7f7c0ef3e640 1 --2- 172.21.15.7:0/2115889690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c081032d0 0x7f7c08109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.669+0000 7f7c0ef3e640 1 --2- 172.21.15.7:0/2115889690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c08102910 0x7f7c08102d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.669+0000 7f7c0ef3e640 1 --2- 172.21.15.7:0/2115889690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c08101720 0x7f7c08101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.669+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/2115889690 >> 172.21.15.7:0/2115889690 conn(0x7f7c080fccb0 msgr2=0x7f7c080ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.669+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/2115889690 shutdown_connections 2024-03-23T13:17:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.669+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/2115889690 wait complete. 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ef3e640 1 Processor -- start 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ef3e640 1 -- start start 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ef3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c08101720 0x7f7c080754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ef3e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c08102910 0x7f7c08073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ef3e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c081032d0 0x7f7c08074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ef3e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c080747a0 con 0x7f7c08101720 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ef3e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08074910 con 0x7f7c081032d0 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ef3e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081a1830 con 0x7f7c08102910 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ccda640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c08101720 0x7f7c080754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ccda640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c08101720 0x7f7c080754c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52478/0 (socket says 172.21.15.7:52478) 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0ccda640 1 -- 172.21.15.7:0/924251907 learned_addr learned my addr 172.21.15.7:0/924251907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c0d4db640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c081032d0 0x7f7c08074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.670+0000 7f7c07fff640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c08102910 0x7f7c08073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c0ccda640 1 -- 172.21.15.7:0/924251907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c08102910 msgr2=0x7f7c08073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c0ccda640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c08102910 0x7f7c08073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c0ccda640 1 -- 172.21.15.7:0/924251907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c081032d0 msgr2=0x7f7c08074040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c0ccda640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c081032d0 0x7f7c08074040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c0ccda640 1 -- 172.21.15.7:0/924251907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf8009900 con 0x7f7c08101720 2024-03-23T13:17:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c0ccda640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c08101720 0x7f7c080754c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000d980 tx=0x7f7c0000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c05ffb640 1 -- 172.21.15.7:0/924251907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00009c30 con 0x7f7c08101720 2024-03-23T13:17:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c05ffb640 1 -- 172.21.15.7:0/924251907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c00009d90 con 0x7f7c08101720 2024-03-23T13:17:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/924251907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c081a1ae0 con 0x7f7c08101720 2024-03-23T13:17:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c05ffb640 1 -- 172.21.15.7:0/924251907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0000f6b0 con 0x7f7c08101720 2024-03-23T13:17:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.671+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/924251907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081a1fc0 con 0x7f7c08101720 2024-03-23T13:17:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.672+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/924251907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c08108ae0 con 0x7f7c08101720 2024-03-23T13:17:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.673+0000 7f7c05ffb640 1 -- 172.21.15.7:0/924251907 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f7c00010460 con 0x7f7c08101720 2024-03-23T13:17:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.674+0000 7f7c05ffb640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7bf40a2140 0x7f7bf40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.674+0000 7f7c07fff640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7bf40a2140 0x7f7bf40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.674+0000 7f7c05ffb640 1 -- 172.21.15.7:0/924251907 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7c000cd8d0 con 0x7f7c08101720 2024-03-23T13:17:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.674+0000 7f7c07fff640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7bf40a2140 0x7f7bf40a4600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8006010 tx=0x7f7bf80326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.678+0000 7f7c05ffb640 1 -- 172.21.15.7:0/924251907 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7c00096a00 con 0x7f7c08101720 2024-03-23T13:17:44.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:44 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3846503166' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T13:17:44.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:44 smithi007 ceph-mon[32184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:44.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:44 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4148330124' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T13:17:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:44.901+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/924251907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f7c0802d230 con 0x7f7c08101720 2024-03-23T13:17:44.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:44 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3846503166' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T13:17:44.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:44 smithi130 ceph-mon[36225]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:44.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:44 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/4148330124' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T13:17:45.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:44 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3846503166' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T13:17:45.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:44 smithi155 ceph-mon[36157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:45.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:44 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/4148330124' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T13:17:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.546+0000 7f7c05ffb640 1 -- 172.21.15.7:0/924251907 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f7c0009a0c0 con 0x7f7c08101720 2024-03-23T13:17:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/924251907 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7bf40a2140 msgr2=0x7f7bf40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7bf40a2140 0x7f7bf40a4600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8006010 tx=0x7f7bf80326b0 comp rx=0 tx=0).stop 2024-03-23T13:17:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/924251907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c08101720 msgr2=0x7f7c080754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:45.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c08101720 0x7f7c080754c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000d980 tx=0x7f7c0000de40 comp rx=0 tx=0).stop 2024-03-23T13:17:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/924251907 shutdown_connections 2024-03-23T13:17:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7bf40a2140 0x7f7bf40a4600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c081032d0 0x7f7c08074040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c08102910 0x7f7c08073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 --2- 172.21.15.7:0/924251907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c08101720 0x7f7c080754c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/924251907 >> 172.21.15.7:0/924251907 conn(0x7f7c080fccb0 msgr2=0x7f7c08105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/924251907 shutdown_connections 2024-03-23T13:17:45.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.549+0000 7f7c0ef3e640 1 -- 172.21.15.7:0/924251907 wait complete. 2024-03-23T13:17:45.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-23T13:17:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 -- 172.21.15.7:0/785962987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdae4102930 msgr2=0x7fdae4102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 --2- 172.21.15.7:0/785962987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdae4102930 0x7fdae4102db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009900 tx=0x7fdad802bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 -- 172.21.15.7:0/785962987 shutdown_connections 2024-03-23T13:17:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 --2- 172.21.15.7:0/785962987 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae41032f0 0x7fdae4109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 --2- 172.21.15.7:0/785962987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdae4102930 0x7fdae4102db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 --2- 172.21.15.7:0/785962987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae4101740 0x7fdae4101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 -- 172.21.15.7:0/785962987 >> 172.21.15.7:0/785962987 conn(0x7fdae40fccb0 msgr2=0x7fdae40ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 -- 172.21.15.7:0/785962987 shutdown_connections 2024-03-23T13:17:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 -- 172.21.15.7:0/785962987 wait complete. 2024-03-23T13:17:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 Processor -- start 2024-03-23T13:17:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 -- start start 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae4101740 0x7fdae41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4102930 0x7fdae4197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdae41032f0 0x7fdae419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdae419f2e0 con 0x7fdae41032f0 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdae419f450 con 0x7fdae4102930 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.678+0000 7fdaebf6c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdae419f610 con 0x7fdae4101740 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaea509640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdae41032f0 0x7fdae419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdae9d08640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae4101740 0x7fdae41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaea509640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdae41032f0 0x7fdae419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52498/0 (socket says 172.21.15.7:52498) 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaea509640 1 -- 172.21.15.7:0/2630791620 learned_addr learned my addr 172.21.15.7:0/2630791620 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdae9507640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4102930 0x7fdae4197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaea509640 1 -- 172.21.15.7:0/2630791620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae4101740 msgr2=0x7fdae41977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaea509640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae4101740 0x7fdae41977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaea509640 1 -- 172.21.15.7:0/2630791620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4102930 msgr2=0x7fdae4197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaea509640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4102930 0x7fdae4197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaea509640 1 -- 172.21.15.7:0/2630791620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad8009560 con 0x7fdae41032f0 2024-03-23T13:17:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaea509640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdae41032f0 0x7fdae419eda0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdad400eb10 tx=0x7fdad400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdae2ffd640 1 -- 172.21.15.7:0/2630791620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4004b20 con 0x7fdae41032f0 2024-03-23T13:17:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdae2ffd640 1 -- 172.21.15.7:0/2630791620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad4004c80 con 0x7fdae41032f0 2024-03-23T13:17:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaebf6c640 1 -- 172.21.15.7:0/2630791620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae419f8f0 con 0x7fdae41032f0 2024-03-23T13:17:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdae2ffd640 1 -- 172.21.15.7:0/2630791620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4010650 con 0x7fdae41032f0 2024-03-23T13:17:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.679+0000 7fdaebf6c640 1 -- 172.21.15.7:0/2630791620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae419fdd0 con 0x7fdae41032f0 2024-03-23T13:17:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.680+0000 7fdaebf6c640 1 -- 172.21.15.7:0/2630791620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae4108a60 con 0x7fdae41032f0 2024-03-23T13:17:45.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.682+0000 7fdae2ffd640 1 -- 172.21.15.7:0/2630791620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7fdad4005020 con 0x7fdae41032f0 2024-03-23T13:17:45.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.682+0000 7fdae2ffd640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdacc0a2140 0x7fdacc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:45.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.682+0000 7fdae9d08640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdacc0a2140 0x7fdacc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:45.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.683+0000 7fdae2ffd640 1 -- 172.21.15.7:0/2630791620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fdad4014070 con 0x7fdae41032f0 2024-03-23T13:17:45.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.683+0000 7fdae9d08640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdacc0a2140 0x7fdacc0a4600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdae4102790 tx=0x7fdadc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.687+0000 7fdae2ffd640 1 -- 172.21.15.7:0/2630791620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fdad40971b0 con 0x7fdae41032f0 2024-03-23T13:17:45.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:45 smithi007 ceph-mon[32184]: pgmap v286: 97 pgs: 18 creating+peering, 23 unknown, 56 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-03-23T13:17:45.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:45 smithi007 ceph-mon[32184]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T13:17:45.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:45 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4148330124' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-23T13:17:45.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:45 smithi007 ceph-mon[32184]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T13:17:45.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:45 smithi007 ceph-mon[32184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:45.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:45 smithi007 ceph-mon[32184]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-23T13:17:45.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:45 smithi007 ceph-mon[32184]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-23T13:17:45.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:45 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/924251907' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T13:17:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:45.908+0000 7fdaebf6c640 1 -- 172.21.15.7:0/2630791620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fdae402d1d0 con 0x7fdae41032f0 2024-03-23T13:17:45.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:45 smithi130 ceph-mon[36225]: pgmap v286: 97 pgs: 18 creating+peering, 23 unknown, 56 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-03-23T13:17:45.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:45 smithi130 ceph-mon[36225]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T13:17:45.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:45 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/4148330124' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-23T13:17:45.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:45 smithi130 ceph-mon[36225]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T13:17:45.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:45 smithi130 ceph-mon[36225]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:45.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:45 smithi130 ceph-mon[36225]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-23T13:17:45.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:45 smithi130 ceph-mon[36225]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-23T13:17:45.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:45 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/924251907' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T13:17:46.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:45 smithi155 ceph-mon[36157]: pgmap v286: 97 pgs: 18 creating+peering, 23 unknown, 56 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-03-23T13:17:46.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:45 smithi155 ceph-mon[36157]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T13:17:46.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:45 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/4148330124' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-23T13:17:46.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:45 smithi155 ceph-mon[36157]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T13:17:46.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:45 smithi155 ceph-mon[36157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T13:17:46.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:45 smithi155 ceph-mon[36157]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-23T13:17:46.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:45 smithi155 ceph-mon[36157]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-23T13:17:46.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:45 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/924251907' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T13:17:46.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.559+0000 7fdae2ffd640 1 -- 172.21.15.7:0/2630791620 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7fdad409a870 con 0x7fdae41032f0 2024-03-23T13:17:46.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.562+0000 7fdaebf6c640 1 -- 172.21.15.7:0/2630791620 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdacc0a2140 msgr2=0x7fdacc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:46.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.562+0000 7fdaebf6c640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdacc0a2140 0x7fdacc0a4600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdae4102790 tx=0x7fdadc005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:46.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.562+0000 7fdaebf6c640 1 -- 172.21.15.7:0/2630791620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdae41032f0 msgr2=0x7fdae419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:46.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.563+0000 7fdaebf6c640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdae41032f0 0x7fdae419eda0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdad400eb10 tx=0x7fdad400c2d0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.563+0000 7fdaebf6c640 1 -- 172.21.15.7:0/2630791620 shutdown_connections 2024-03-23T13:17:46.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.563+0000 7fdaebf6c640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdacc0a2140 0x7fdacc0a4600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.563+0000 7fdaebf6c640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdae41032f0 0x7fdae419eda0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.563+0000 7fdaebf6c640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4102930 0x7fdae4197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.563+0000 7fdaebf6c640 1 --2- 172.21.15.7:0/2630791620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae4101740 0x7fdae41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.563+0000 7fdaebf6c640 1 -- 172.21.15.7:0/2630791620 >> 172.21.15.7:0/2630791620 conn(0x7fdae40fccb0 msgr2=0x7fdae4105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:46.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.563+0000 7fdaebf6c640 1 -- 172.21.15.7:0/2630791620 shutdown_connections 2024-03-23T13:17:46.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.563+0000 7fdaebf6c640 1 -- 172.21.15.7:0/2630791620 wait complete. 2024-03-23T13:17:46.579 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-23T13:17:46.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.691+0000 7f9233215640 1 -- 172.21.15.7:0/2751448106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f922c1017a0 msgr2=0x7f922c101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.691+0000 7f9233215640 1 --2- 172.21.15.7:0/2751448106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f922c1017a0 0x7f922c101bc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9224009ca0 tx=0x7f922402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.691+0000 7f9233215640 1 -- 172.21.15.7:0/2751448106 shutdown_connections 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.691+0000 7f9233215640 1 --2- 172.21.15.7:0/2751448106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f922c103350 0x7f922c109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.691+0000 7f9233215640 1 --2- 172.21.15.7:0/2751448106 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f922c102990 0x7f922c102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.691+0000 7f9233215640 1 --2- 172.21.15.7:0/2751448106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f922c1017a0 0x7f922c101bc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.691+0000 7f9233215640 1 -- 172.21.15.7:0/2751448106 >> 172.21.15.7:0/2751448106 conn(0x7f922c0fcd30 msgr2=0x7f922c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9233215640 1 -- 172.21.15.7:0/2751448106 shutdown_connections 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9233215640 1 -- 172.21.15.7:0/2751448106 wait complete. 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9233215640 1 Processor -- start 2024-03-23T13:17:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9233215640 1 -- start start 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9233215640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f922c1017a0 0x7f922c0754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9233215640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f922c102990 0x7f922c073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9233215640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f922c103350 0x7f922c074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9233215640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f922c0747a0 con 0x7f922c1017a0 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9233215640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f922c074910 con 0x7f922c103350 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9233215640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f922c1a1800 con 0x7f922c102990 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9230fb1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f922c1017a0 0x7f922c0754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9230fb1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f922c1017a0 0x7f922c0754c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52524/0 (socket says 172.21.15.7:52524) 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f9230fb1640 1 -- 172.21.15.7:0/698054134 learned_addr learned my addr 172.21.15.7:0/698054134 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f922bfff640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f922c102990 0x7f922c073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.692+0000 7f92317b2640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f922c103350 0x7f922c074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9230fb1640 1 -- 172.21.15.7:0/698054134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f922c102990 msgr2=0x7f922c073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9230fb1640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f922c102990 0x7f922c073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9230fb1640 1 -- 172.21.15.7:0/698054134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f922c103350 msgr2=0x7f922c074040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9230fb1640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f922c103350 0x7f922c074040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9230fb1640 1 -- 172.21.15.7:0/698054134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9224009900 con 0x7f922c1017a0 2024-03-23T13:17:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9230fb1640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f922c1017a0 0x7f922c0754c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9224037040 tx=0x7f9224032fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9229ffb640 1 -- 172.21.15.7:0/698054134 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224034e20 con 0x7f922c1017a0 2024-03-23T13:17:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9229ffb640 1 -- 172.21.15.7:0/698054134 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9224003e50 con 0x7f922c1017a0 2024-03-23T13:17:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9233215640 1 -- 172.21.15.7:0/698054134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922c1a1a50 con 0x7f922c1017a0 2024-03-23T13:17:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9229ffb640 1 -- 172.21.15.7:0/698054134 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224040450 con 0x7f922c1017a0 2024-03-23T13:17:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.693+0000 7f9233215640 1 -- 172.21.15.7:0/698054134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f922c1a1f50 con 0x7f922c1017a0 2024-03-23T13:17:46.696 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:46 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/924251907' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T13:17:46.696 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:46 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] up:active 2024-03-23T13:17:46.696 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:46 smithi130 ceph-mon[36225]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T13:17:46.696 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:46 smithi130 ceph-mon[36225]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-23T13:17:46.696 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:46 smithi130 ceph-mon[36225]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-03-23T13:17:46.696 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:46 smithi130 ceph-mon[36225]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-23T13:17:46.696 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:46 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2630791620' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T13:17:46.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.694+0000 7f9233215640 1 -- 172.21.15.7:0/698054134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922c108b40 con 0x7f922c1017a0 2024-03-23T13:17:46.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.695+0000 7f9229ffb640 1 -- 172.21.15.7:0/698054134 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f922400a040 con 0x7f922c1017a0 2024-03-23T13:17:46.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.696+0000 7f9229ffb640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92180a2140 0x7f92180a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:46.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.696+0000 7f922bfff640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92180a2140 0x7f92180a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:46.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.696+0000 7f9229ffb640 1 -- 172.21.15.7:0/698054134 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f92240f0cb0 con 0x7f922c1017a0 2024-03-23T13:17:46.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.696+0000 7f922bfff640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92180a2140 0x7f92180a4600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f922c075230 tx=0x7f921c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:46.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.700+0000 7f9229ffb640 1 -- 172.21.15.7:0/698054134 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f92240b9db0 con 0x7f922c1017a0 2024-03-23T13:17:46.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/924251907' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T13:17:46.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] up:active 2024-03-23T13:17:46.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-mon[32184]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T13:17:46.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-mon[32184]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-23T13:17:46.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-mon[32184]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-03-23T13:17:46.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-mon[32184]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-23T13:17:46.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2630791620' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T13:17:46.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:46.909+0000 7f9233215640 1 -- 172.21.15.7:0/698054134 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f922c02d240 con 0x7f92180a2140 2024-03-23T13:17:46.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:46 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/924251907' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T13:17:46.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:46 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] up:active 2024-03-23T13:17:46.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:46 smithi155 ceph-mon[36157]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T13:17:46.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:46 smithi155 ceph-mon[36157]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-23T13:17:46.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:46 smithi155 ceph-mon[36157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-03-23T13:17:46.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:46 smithi155 ceph-mon[36157]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-23T13:17:46.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:46 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2630791620' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T13:17:47.326 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:17:46.913+0000 7fb59b2a5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T13:17:47.350 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:17:46.913+0000 7fb59b2a5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T13:17:47.350 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:17:46.913+0000 7fb59b2a5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T13:17:47.350 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:17:46.913+0000 7fb59b2a5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T13:17:47.350 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:17:46 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:17:46.913+0000 7fb59b2a5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T13:17:47.559 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: 2024-03-23T13:17:47.201+0000 7fead4ed0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T13:17:47.559 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: 2024-03-23T13:17:47.201+0000 7fead4ed0700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-23T13:17:47.559 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: 2024-03-23T13:17:47.201+0000 7fead4ed0700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-23T13:17:47.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:47 smithi007 ceph-mon[32184]: pgmap v287: 97 pgs: 2 creating+peering, 95 active+clean; 580 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2024-03-23T13:17:47.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:47 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2630791620' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T13:17:47.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:47 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] up:active 2024-03-23T13:17:47.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:47 smithi007 ceph-mon[32184]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-23T13:17:47.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:47 smithi007 ceph-mon[32184]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-03-23T13:17:47.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:47 smithi007 ceph-mon[32184]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-03-23T13:17:47.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:47 smithi007 ceph-mon[32184]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-23T13:17:47.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:47 smithi130 ceph-mon[36225]: pgmap v287: 97 pgs: 2 creating+peering, 95 active+clean; 580 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2024-03-23T13:17:47.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:47 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2630791620' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T13:17:47.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:47 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] up:active 2024-03-23T13:17:47.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:47 smithi130 ceph-mon[36225]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-23T13:17:47.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:47 smithi130 ceph-mon[36225]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-03-23T13:17:47.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:47 smithi130 ceph-mon[36225]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-03-23T13:17:47.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:47 smithi130 ceph-mon[36225]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-23T13:17:47.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:47 smithi155 ceph-mon[36157]: pgmap v287: 97 pgs: 2 creating+peering, 95 active+clean; 580 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2024-03-23T13:17:47.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:47 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2630791620' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T13:17:47.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:47 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] up:active 2024-03-23T13:17:47.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:47 smithi155 ceph-mon[36157]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-23T13:17:47.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:47 smithi155 ceph-mon[36157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-03-23T13:17:47.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:47 smithi155 ceph-mon[36157]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-03-23T13:17:47.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:47 smithi155 ceph-mon[36157]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-23T13:17:48.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.278+0000 7f9229ffb640 1 -- 172.21.15.7:0/698054134 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f922c02d240 con 0x7f92180a2140 2024-03-23T13:17:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.280+0000 7f9233215640 1 -- 172.21.15.7:0/698054134 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92180a2140 msgr2=0x7f92180a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92180a2140 0x7f92180a4600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f922c075230 tx=0x7f921c005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 -- 172.21.15.7:0/698054134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f922c1017a0 msgr2=0x7f922c0754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f922c1017a0 0x7f922c0754c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9224037040 tx=0x7f9224032fa0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 -- 172.21.15.7:0/698054134 shutdown_connections 2024-03-23T13:17:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92180a2140 0x7f92180a4600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f922c103350 0x7f922c074040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f922c102990 0x7f922c073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 --2- 172.21.15.7:0/698054134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f922c1017a0 0x7f922c0754c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 -- 172.21.15.7:0/698054134 >> 172.21.15.7:0/698054134 conn(0x7f922c0fcd30 msgr2=0x7f922c105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 -- 172.21.15.7:0/698054134 shutdown_connections 2024-03-23T13:17:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.281+0000 7f9233215640 1 -- 172.21.15.7:0/698054134 wait complete. 2024-03-23T13:17:48.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-23T13:17:48.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.409+0000 7f971cd81640 1 -- 172.21.15.7:0/2441294876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9718102910 msgr2=0x7f9718102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.409+0000 7f971cd81640 1 --2- 172.21.15.7:0/2441294876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9718102910 0x7f9718102d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9700009900 tx=0x7f970002bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.409+0000 7f971cd81640 1 -- 172.21.15.7:0/2441294876 shutdown_connections 2024-03-23T13:17:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.409+0000 7f971cd81640 1 --2- 172.21.15.7:0/2441294876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97181032d0 0x7f9718109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.409+0000 7f971cd81640 1 --2- 172.21.15.7:0/2441294876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9718102910 0x7f9718102d90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.409+0000 7f971cd81640 1 --2- 172.21.15.7:0/2441294876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9718101720 0x7f9718101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.409+0000 7f971cd81640 1 -- 172.21.15.7:0/2441294876 >> 172.21.15.7:0/2441294876 conn(0x7f97180fccb0 msgr2=0x7f97180ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.409+0000 7f971cd81640 1 -- 172.21.15.7:0/2441294876 shutdown_connections 2024-03-23T13:17:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.409+0000 7f971cd81640 1 -- 172.21.15.7:0/2441294876 wait complete. 2024-03-23T13:17:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971cd81640 1 Processor -- start 2024-03-23T13:17:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971cd81640 1 -- start start 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971cd81640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9718101720 0x7f97181977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971cd81640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9718102910 0x7f9718197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971cd81640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97181032d0 0x7f971819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971cd81640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f971819f2c0 con 0x7f9718101720 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971cd81640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f971819f430 con 0x7f9718102910 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971cd81640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f971819f5f0 con 0x7f97181032d0 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9718101720 0x7f97181977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9718101720 0x7f97181977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52560/0 (socket says 172.21.15.7:52560) 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971659c640 1 -- 172.21.15.7:0/2869602725 learned_addr learned my addr 172.21.15.7:0/2869602725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f9716d9d640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97181032d0 0x7f971819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f9715d9b640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9718102910 0x7f9718197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971659c640 1 -- 172.21.15.7:0/2869602725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97181032d0 msgr2=0x7f971819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971659c640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97181032d0 0x7f971819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971659c640 1 -- 172.21.15.7:0/2869602725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9718102910 msgr2=0x7f9718197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971659c640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9718102910 0x7f9718197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971659c640 1 -- 172.21.15.7:0/2869602725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9700009560 con 0x7f9718101720 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.410+0000 7f971659c640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9718101720 0x7f97181977d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f971000d950 tx=0x7f971000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.411+0000 7f970f7fe640 1 -- 172.21.15.7:0/2869602725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710009c30 con 0x7f9718101720 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.411+0000 7f970f7fe640 1 -- 172.21.15.7:0/2869602725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9710009d90 con 0x7f9718101720 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.411+0000 7f971cd81640 1 -- 172.21.15.7:0/2869602725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971819f8d0 con 0x7f9718101720 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.411+0000 7f970f7fe640 1 -- 172.21.15.7:0/2869602725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971000f850 con 0x7f9718101720 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.411+0000 7f971cd81640 1 -- 172.21.15.7:0/2869602725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971819fdb0 con 0x7f9718101720 2024-03-23T13:17:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.412+0000 7f971cd81640 1 -- 172.21.15.7:0/2869602725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9718108ac0 con 0x7f9718101720 2024-03-23T13:17:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.413+0000 7f970f7fe640 1 -- 172.21.15.7:0/2869602725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f971000f9b0 con 0x7f9718101720 2024-03-23T13:17:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.414+0000 7f970f7fe640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f97040a20c0 0x7f97040a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.414+0000 7f9715d9b640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f97040a20c0 0x7f97040a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.414+0000 7f970f7fe640 1 -- 172.21.15.7:0/2869602725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f97100cdab0 con 0x7f9718101720 2024-03-23T13:17:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.414+0000 7f9715d9b640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f97040a20c0 0x7f97040a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9700009230 tx=0x7f9700005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.418+0000 7f970f7fe640 1 -- 172.21.15.7:0/2869602725 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9710096bd0 con 0x7f9718101720 2024-03-23T13:17:48.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.629+0000 7f971cd81640 1 -- 172.21.15.7:0/2869602725 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f971802d2a0 con 0x7f97040a20c0 2024-03-23T13:17:48.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.652+0000 7f970f7fe640 1 -- 172.21.15.7:0/2869602725 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f971802d2a0 con 0x7f97040a20c0 2024-03-23T13:17:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 -- 172.21.15.7:0/2869602725 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f97040a20c0 msgr2=0x7f97040a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f97040a20c0 0x7f97040a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9700009230 tx=0x7f9700005fd0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 -- 172.21.15.7:0/2869602725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9718101720 msgr2=0x7f97181977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9718101720 0x7f97181977d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f971000d950 tx=0x7f971000de10 comp rx=0 tx=0).stop 2024-03-23T13:17:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 -- 172.21.15.7:0/2869602725 shutdown_connections 2024-03-23T13:17:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f97040a20c0 0x7f97040a4580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97181032d0 0x7f971819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9718102910 0x7f9718197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 --2- 172.21.15.7:0/2869602725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9718101720 0x7f97181977d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 -- 172.21.15.7:0/2869602725 >> 172.21.15.7:0/2869602725 conn(0x7f97180fccb0 msgr2=0x7f9718105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 -- 172.21.15.7:0/2869602725 shutdown_connections 2024-03-23T13:17:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.654+0000 7f971cd81640 1 -- 172.21.15.7:0/2869602725 wait complete. 2024-03-23T13:17:48.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-23T13:17:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.773+0000 7f3aad60e640 1 -- 172.21.15.7:0/1743732661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3aa8102910 msgr2=0x7f3aa8102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.773+0000 7f3aad60e640 1 --2- 172.21.15.7:0/1743732661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3aa8102910 0x7f3aa8102d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009900 tx=0x7f3a9002bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.773+0000 7f3aad60e640 1 -- 172.21.15.7:0/1743732661 shutdown_connections 2024-03-23T13:17:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.773+0000 7f3aad60e640 1 --2- 172.21.15.7:0/1743732661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa81032d0 0x7f3aa8109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.773+0000 7f3aad60e640 1 --2- 172.21.15.7:0/1743732661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3aa8102910 0x7f3aa8102d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.773+0000 7f3aad60e640 1 --2- 172.21.15.7:0/1743732661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa8101720 0x7f3aa8101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.773+0000 7f3aad60e640 1 -- 172.21.15.7:0/1743732661 >> 172.21.15.7:0/1743732661 conn(0x7f3aa80fccb0 msgr2=0x7f3aa80ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aad60e640 1 -- 172.21.15.7:0/1743732661 shutdown_connections 2024-03-23T13:17:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aad60e640 1 -- 172.21.15.7:0/1743732661 wait complete. 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aad60e640 1 Processor -- start 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aad60e640 1 -- start start 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aad60e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa8101720 0x7f3aa80754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aad60e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3aa8102910 0x7f3aa8073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aad60e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa81032d0 0x7f3aa8074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aad60e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa80747a0 con 0x7f3aa8102910 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aad60e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa8074910 con 0x7f3aa81032d0 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aad60e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81a1880 con 0x7f3aa8101720 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3a9ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3aa8102910 0x7f3aa8073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3a9ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3aa8102910 0x7f3aa8073b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52576/0 (socket says 172.21.15.7:52576) 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3a9ffff640 1 -- 172.21.15.7:0/1724059619 learned_addr learned my addr 172.21.15.7:0/1724059619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aa6ffd640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa8101720 0x7f3aa80754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.774+0000 7f3aa77fe640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa81032d0 0x7f3aa8074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3a9ffff640 1 -- 172.21.15.7:0/1724059619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa8101720 msgr2=0x7f3aa80754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3a9ffff640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa8101720 0x7f3aa80754c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3a9ffff640 1 -- 172.21.15.7:0/1724059619 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa81032d0 msgr2=0x7f3aa8074040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3a9ffff640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa81032d0 0x7f3aa8074040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3a9ffff640 1 -- 172.21.15.7:0/1724059619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a90009560 con 0x7f3aa8102910 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3a9ffff640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3aa8102910 0x7f3aa8073b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3a90000c00 tx=0x7f3a900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3aa4ff9640 1 -- 172.21.15.7:0/1724059619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a900044e0 con 0x7f3aa8102910 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3aad60e640 1 -- 172.21.15.7:0/1724059619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa81a1ad0 con 0x7f3aa8102910 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3aa4ff9640 1 -- 172.21.15.7:0/1724059619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a90004ac0 con 0x7f3aa8102910 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3aa4ff9640 1 -- 172.21.15.7:0/1724059619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9003e450 con 0x7f3aa8102910 2024-03-23T13:17:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.775+0000 7f3aad60e640 1 -- 172.21.15.7:0/1724059619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa81a2030 con 0x7f3aa8102910 2024-03-23T13:17:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.776+0000 7f3aad60e640 1 -- 172.21.15.7:0/1724059619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa8108ac0 con 0x7f3aa8102910 2024-03-23T13:17:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.777+0000 7f3aa4ff9640 1 -- 172.21.15.7:0/1724059619 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f3a90003680 con 0x7f3aa8102910 2024-03-23T13:17:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.778+0000 7f3aa4ff9640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3a940a20c0 0x7f3a940a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.778+0000 7f3aa6ffd640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3a940a20c0 0x7f3a940a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.778+0000 7f3aa4ff9640 1 -- 172.21.15.7:0/1724059619 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3a900efea0 con 0x7f3aa8102910 2024-03-23T13:17:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.778+0000 7f3aa6ffd640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3a940a20c0 0x7f3a940a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8102770 tx=0x7f3a98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.782+0000 7f3aa4ff9640 1 -- 172.21.15.7:0/1724059619 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f3a900b8f20 con 0x7f3aa8102910 2024-03-23T13:17:48.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:48 smithi007 ceph-mon[32184]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:48.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:48 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] up:active 2024-03-23T13:17:48.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:48 smithi007 ceph-mon[32184]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-23T13:17:48.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:48 smithi007 ceph-mon[32184]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T13:17:48.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:48 smithi007 ceph-mon[32184]: Cluster is now healthy 2024-03-23T13:17:48.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:48 smithi007 ceph-mon[32184]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-03-23T13:17:48.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:48 smithi007 ceph-mon[32184]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-23T13:17:48.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:48 smithi007 ceph-mon[32184]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-23T13:17:48.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:48 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] up:active 2024-03-23T13:17:48.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:48 smithi007 ceph-mon[32184]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:17:48.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:48 smithi130 ceph-mon[36225]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:48.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:48 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] up:active 2024-03-23T13:17:48.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:48 smithi130 ceph-mon[36225]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-23T13:17:48.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:48 smithi130 ceph-mon[36225]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T13:17:48.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:48 smithi130 ceph-mon[36225]: Cluster is now healthy 2024-03-23T13:17:48.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:48 smithi130 ceph-mon[36225]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-03-23T13:17:48.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:48 smithi130 ceph-mon[36225]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-23T13:17:48.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:48 smithi130 ceph-mon[36225]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-23T13:17:48.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:48 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] up:active 2024-03-23T13:17:48.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:48 smithi130 ceph-mon[36225]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:17:48.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.995+0000 7f3aad60e640 1 -- 172.21.15.7:0/1724059619 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3aa802d240 con 0x7f3a940a20c0 2024-03-23T13:17:48.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:48.999+0000 7f3aa4ff9640 1 -- 172.21.15.7:0/1724059619 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3aa802d240 con 0x7f3a940a20c0 2024-03-23T13:17:48.999 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_0/aab66750-01c4-45ff-9b8c-3fb79be121ca 2024-03-23T13:17:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 -- 172.21.15.7:0/1724059619 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3a940a20c0 msgr2=0x7f3a940a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3a940a20c0 0x7f3a940a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8102770 tx=0x7f3a98005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 -- 172.21.15.7:0/1724059619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3aa8102910 msgr2=0x7f3aa8073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3aa8102910 0x7f3aa8073b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3a90000c00 tx=0x7f3a900042f0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 -- 172.21.15.7:0/1724059619 shutdown_connections 2024-03-23T13:17:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3a940a20c0 0x7f3a940a4580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa81032d0 0x7f3aa8074040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3aa8102910 0x7f3aa8073b00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 --2- 172.21.15.7:0/1724059619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa8101720 0x7f3aa80754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 -- 172.21.15.7:0/1724059619 >> 172.21.15.7:0/1724059619 conn(0x7f3aa80fccb0 msgr2=0x7f3aa8105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 -- 172.21.15.7:0/1724059619 shutdown_connections 2024-03-23T13:17:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.001+0000 7f3aad60e640 1 -- 172.21.15.7:0/1724059619 wait complete. 2024-03-23T13:17:49.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-23T13:17:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:48 smithi155 ceph-mon[36157]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:48 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] up:active 2024-03-23T13:17:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:48 smithi155 ceph-mon[36157]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-23T13:17:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:48 smithi155 ceph-mon[36157]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T13:17:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:48 smithi155 ceph-mon[36157]: Cluster is now healthy 2024-03-23T13:17:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:48 smithi155 ceph-mon[36157]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-03-23T13:17:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:48 smithi155 ceph-mon[36157]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-23T13:17:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:48 smithi155 ceph-mon[36157]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-23T13:17:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:48 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] up:active 2024-03-23T13:17:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:48 smithi155 ceph-mon[36157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:17:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 -- 172.21.15.7:0/672513654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81a8101820 msgr2=0x7f81a8101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 --2- 172.21.15.7:0/672513654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81a8101820 0x7f81a8101ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f819c009900 tx=0x7f819c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 -- 172.21.15.7:0/672513654 shutdown_connections 2024-03-23T13:17:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 --2- 172.21.15.7:0/672513654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a81021e0 0x7f81a8108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 --2- 172.21.15.7:0/672513654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81a8101820 0x7f81a8101ca0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 --2- 172.21.15.7:0/672513654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a8100630 0x7f81a8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 -- 172.21.15.7:0/672513654 >> 172.21.15.7:0/672513654 conn(0x7f81a80fbbc0 msgr2=0x7f81a80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 -- 172.21.15.7:0/672513654 shutdown_connections 2024-03-23T13:17:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 -- 172.21.15.7:0/672513654 wait complete. 2024-03-23T13:17:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 Processor -- start 2024-03-23T13:17:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.119+0000 7f81ae5e2640 1 -- start start 2024-03-23T13:17:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81ae5e2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8100630 0x7f81a806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81ae5e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81a8101820 0x7f81a819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81ae5e2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a81021e0 0x7f81a819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81ae5e2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81a819f2c0 con 0x7f81a8101820 2024-03-23T13:17:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81ae5e2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f81a819f430 con 0x7f81a81021e0 2024-03-23T13:17:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81ae5e2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f81a819f5f0 con 0x7f81a8100630 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81a8101820 0x7f81a819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81a8101820 0x7f81a819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52594/0 (socket says 172.21.15.7:52594) 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a77fe640 1 -- 172.21.15.7:0/2346785348 learned_addr learned my addr 172.21.15.7:0/2346785348 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a7fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8100630 0x7f81a806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81acb7f640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a81021e0 0x7f81a819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a77fe640 1 -- 172.21.15.7:0/2346785348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8100630 msgr2=0x7f81a806a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a77fe640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8100630 0x7f81a806a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a77fe640 1 -- 172.21.15.7:0/2346785348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a81021e0 msgr2=0x7f81a819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a77fe640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a81021e0 0x7f81a819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a77fe640 1 -- 172.21.15.7:0/2346785348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c009560 con 0x7f81a8101820 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a77fe640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81a8101820 0x7f81a819c4c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f819c006010 tx=0x7f819c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.120+0000 7f81a57fa640 1 -- 172.21.15.7:0/2346785348 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c0043d0 con 0x7f81a8101820 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.121+0000 7f81a57fa640 1 -- 172.21.15.7:0/2346785348 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819c0052d0 con 0x7f81a8101820 2024-03-23T13:17:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.121+0000 7f81ae5e2640 1 -- 172.21.15.7:0/2346785348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a819f870 con 0x7f81a8101820 2024-03-23T13:17:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.121+0000 7f81a57fa640 1 -- 172.21.15.7:0/2346785348 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c003cc0 con 0x7f81a8101820 2024-03-23T13:17:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.121+0000 7f81ae5e2640 1 -- 172.21.15.7:0/2346785348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a819fdd0 con 0x7f81a8101820 2024-03-23T13:17:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.121+0000 7f81ae5e2640 1 -- 172.21.15.7:0/2346785348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a8107950 con 0x7f81a8101820 2024-03-23T13:17:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.123+0000 7f81a57fa640 1 -- 172.21.15.7:0/2346785348 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 136755+0+0 (secure 0 0 0) 0x7f819c005440 con 0x7f81a8101820 2024-03-23T13:17:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.123+0000 7f81a57fa640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f81900a2140 0x7f81900a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.123+0000 7f81a7fff640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f81900a2140 0x7f81900a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.123+0000 7f81a57fa640 1 -- 172.21.15.7:0/2346785348 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f819c0f0270 con 0x7f81a8101820 2024-03-23T13:17:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.124+0000 7f81a7fff640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f81900a2140 0x7f81900a4600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f81a8101680 tx=0x7f8198005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.127+0000 7f81a57fa640 1 -- 172.21.15.7:0/2346785348 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f819c0b9390 con 0x7f81a8101820 2024-03-23T13:17:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.274+0000 7f81a57fa640 1 -- 172.21.15.7:0/2346785348 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f819c0b9760 con 0x7f81a8101820 2024-03-23T13:17:49.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.344+0000 7f81ae5e2640 1 -- 172.21.15.7:0/2346785348 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f81a802d240 con 0x7f81900a2140 2024-03-23T13:17:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.369+0000 7f81a57fa640 1 -- 172.21.15.7:0/2346785348 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f81a802d240 con 0x7f81900a2140 2024-03-23T13:17:49.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 -- 172.21.15.7:0/2346785348 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f81900a2140 msgr2=0x7f81900a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f81900a2140 0x7f81900a4600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f81a8101680 tx=0x7f8198005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 -- 172.21.15.7:0/2346785348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81a8101820 msgr2=0x7f81a819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81a8101820 0x7f81a819c4c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f819c006010 tx=0x7f819c005e80 comp rx=0 tx=0).stop 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 -- 172.21.15.7:0/2346785348 shutdown_connections 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f81900a2140 0x7f81900a4600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a81021e0 0x7f81a819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81a8101820 0x7f81a819c4c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 --2- 172.21.15.7:0/2346785348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8100630 0x7f81a806a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 -- 172.21.15.7:0/2346785348 >> 172.21.15.7:0/2346785348 conn(0x7f81a80fbbc0 msgr2=0x7f81a8104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 -- 172.21.15.7:0/2346785348 shutdown_connections 2024-03-23T13:17:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.371+0000 7f81ae5e2640 1 -- 172.21.15.7:0/2346785348 wait complete. 2024-03-23T13:17:49.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-23T13:17:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.489+0000 7f0fa4793640 1 -- 172.21.15.7:0/2965484318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f9c1021c0 msgr2=0x7f0f9c108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.489+0000 7f0fa4793640 1 --2- 172.21.15.7:0/2965484318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f9c1021c0 0x7f0f9c108a50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009ca0 tx=0x7f0f9402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.489+0000 7f0fa4793640 1 -- 172.21.15.7:0/2965484318 shutdown_connections 2024-03-23T13:17:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.489+0000 7f0fa4793640 1 --2- 172.21.15.7:0/2965484318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f9c1021c0 0x7f0f9c108a50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.489+0000 7f0fa4793640 1 --2- 172.21.15.7:0/2965484318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c101800 0x7f0f9c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.489+0000 7f0fa4793640 1 --2- 172.21.15.7:0/2965484318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f9c100610 0x7f0f9c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.489+0000 7f0fa4793640 1 -- 172.21.15.7:0/2965484318 >> 172.21.15.7:0/2965484318 conn(0x7f0f9c0fbba0 msgr2=0x7f0f9c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa4793640 1 -- 172.21.15.7:0/2965484318 shutdown_connections 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa4793640 1 -- 172.21.15.7:0/2965484318 wait complete. 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa4793640 1 Processor -- start 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa4793640 1 -- start start 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa4793640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f9c100610 0x7f0f9c1045a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa4793640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c101800 0x7f0f9c102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa4793640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f9c1021c0 0x7f0f9c103120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa4793640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c103880 con 0x7f0f9c100610 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa4793640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1039f0 con 0x7f0f9c1021c0 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa4793640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1a1860 con 0x7f0f9c101800 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa252f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f9c100610 0x7f0f9c1045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa252f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f9c100610 0x7f0f9c1045a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52614/0 (socket says 172.21.15.7:52614) 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa252f640 1 -- 172.21.15.7:0/1120698272 learned_addr learned my addr 172.21.15.7:0/1120698272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa1d2e640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c101800 0x7f0f9c102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.490+0000 7f0fa2d30640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f9c1021c0 0x7f0f9c103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0fa252f640 1 -- 172.21.15.7:0/1120698272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c101800 msgr2=0x7f0f9c102be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0fa252f640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c101800 0x7f0f9c102be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0fa252f640 1 -- 172.21.15.7:0/1120698272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f9c1021c0 msgr2=0x7f0f9c103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0fa252f640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f9c1021c0 0x7f0f9c103120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0fa252f640 1 -- 172.21.15.7:0/1120698272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f94009900 con 0x7f0f9c100610 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0fa252f640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f9c100610 0x7f0f9c1045a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800e9f0 tx=0x7f0f9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0f937fe640 1 -- 172.21.15.7:0/1120698272 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98009700 con 0x7f0f9c100610 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0f937fe640 1 -- 172.21.15.7:0/1120698272 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9800ccc0 con 0x7f0f9c100610 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0fa4793640 1 -- 172.21.15.7:0/1120698272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c1a1b10 con 0x7f0f9c100610 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0f937fe640 1 -- 172.21.15.7:0/1120698272 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98010430 con 0x7f0f9c100610 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.491+0000 7f0fa4793640 1 -- 172.21.15.7:0/1120698272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c1a2020 con 0x7f0f9c100610 2024-03-23T13:17:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.492+0000 7f0fa4793640 1 -- 172.21.15.7:0/1120698272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f9c10c060 con 0x7f0f9c100610 2024-03-23T13:17:49.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.493+0000 7f0f937fe640 1 -- 172.21.15.7:0/1120698272 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f0f980035f0 con 0x7f0f9c100610 2024-03-23T13:17:49.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.493+0000 7f0f937fe640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0f880a2150 0x7f0f880a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.494+0000 7f0fa1d2e640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0f880a2150 0x7f0f880a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.494+0000 7f0f937fe640 1 -- 172.21.15.7:0/1120698272 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f0f98014070 con 0x7f0f9c100610 2024-03-23T13:17:49.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.494+0000 7f0fa1d2e640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0f880a2150 0x7f0f880a4610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c104310 tx=0x7f0f8c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:49.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.498+0000 7f0f937fe640 1 -- 172.21.15.7:0/1120698272 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f0f98097000 con 0x7f0f9c100610 2024-03-23T13:17:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.709+0000 7f0fa4793640 1 -- 172.21.15.7:0/1120698272 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0f9c0fdd40 con 0x7f0f880a2150 2024-03-23T13:17:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.715+0000 7f0f937fe640 1 -- 172.21.15.7:0/1120698272 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0f9c0fdd40 con 0x7f0f880a2150 2024-03-23T13:17:49.715 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_1/dd5ff27a-26e2-40d7-8415-18d36aa3eebc 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 -- 172.21.15.7:0/1120698272 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0f880a2150 msgr2=0x7f0f880a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0f880a2150 0x7f0f880a4610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c104310 tx=0x7f0f8c009110 comp rx=0 tx=0).stop 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 -- 172.21.15.7:0/1120698272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f9c100610 msgr2=0x7f0f9c1045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f9c100610 0x7f0f9c1045a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800e9f0 tx=0x7f0f9800edf0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 -- 172.21.15.7:0/1120698272 shutdown_connections 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0f880a2150 0x7f0f880a4610 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f9c1021c0 0x7f0f9c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9c101800 0x7f0f9c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 --2- 172.21.15.7:0/1120698272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f9c100610 0x7f0f9c1045a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 -- 172.21.15.7:0/1120698272 >> 172.21.15.7:0/1120698272 conn(0x7f0f9c0fbba0 msgr2=0x7f0f9c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 -- 172.21.15.7:0/1120698272 shutdown_connections 2024-03-23T13:17:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.717+0000 7f0fa4793640 1 -- 172.21.15.7:0/1120698272 wait complete. 2024-03-23T13:17:49.726 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:49 smithi007 ceph-mon[32184]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s wr, 19 op/s 2024-03-23T13:17:49.726 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:49 smithi007 ceph-mon[32184]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:49.726 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:49 smithi007 ceph-mon[32184]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-23T13:17:49.727 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:17:49.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:49 smithi155 ceph-mon[36157]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s wr, 19 op/s 2024-03-23T13:17:49.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:49 smithi155 ceph-mon[36157]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:49.787 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:49 smithi155 ceph-mon[36157]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-23T13:17:49.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.830+0000 7fce6ca47640 1 -- 172.21.15.7:0/318560775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68101820 msgr2=0x7fce68101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.830+0000 7fce6ca47640 1 --2- 172.21.15.7:0/318560775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68101820 0x7fce68101ca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009900 tx=0x7fce5c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:49.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.830+0000 7fce6ca47640 1 -- 172.21.15.7:0/318560775 shutdown_connections 2024-03-23T13:17:49.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.830+0000 7fce6ca47640 1 --2- 172.21.15.7:0/318560775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce681021e0 0x7fce68108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.830+0000 7fce6ca47640 1 --2- 172.21.15.7:0/318560775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68101820 0x7fce68101ca0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.830+0000 7fce6ca47640 1 --2- 172.21.15.7:0/318560775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce68100630 0x7fce68100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.830+0000 7fce6ca47640 1 -- 172.21.15.7:0/318560775 >> 172.21.15.7:0/318560775 conn(0x7fce680fbba0 msgr2=0x7fce680fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:49.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.831+0000 7fce6ca47640 1 -- 172.21.15.7:0/318560775 shutdown_connections 2024-03-23T13:17:49.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.831+0000 7fce6ca47640 1 -- 172.21.15.7:0/318560775 wait complete. 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.831+0000 7fce6ca47640 1 Processor -- start 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.831+0000 7fce6ca47640 1 -- start start 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.831+0000 7fce6ca47640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68100630 0x7fce681999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.831+0000 7fce6ca47640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce68101820 0x7fce68199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.831+0000 7fce6ca47640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce681021e0 0x7fce681a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.831+0000 7fce6ca47640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce681a14b0 con 0x7fce68100630 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.831+0000 7fce6ca47640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce681a1620 con 0x7fce68101820 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.831+0000 7fce6ca47640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce681a17e0 con 0x7fce681021e0 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68100630 0x7fce681999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68100630 0x7fce681999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52636/0 (socket says 172.21.15.7:52636) 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6659c640 1 -- 172.21.15.7:0/1539653978 learned_addr learned my addr 172.21.15.7:0/1539653978 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce66d9d640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce681021e0 0x7fce681a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce65d9b640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce68101820 0x7fce68199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6659c640 1 -- 172.21.15.7:0/1539653978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce681021e0 msgr2=0x7fce681a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6659c640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce681021e0 0x7fce681a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6659c640 1 -- 172.21.15.7:0/1539653978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce68101820 msgr2=0x7fce68199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6659c640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce68101820 0x7fce68199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6659c640 1 -- 172.21.15.7:0/1539653978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c009560 con 0x7fce68100630 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6659c640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68100630 0x7fce681999c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9c0 tx=0x7fce6000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce577fe640 1 -- 172.21.15.7:0/1539653978 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce600096e0 con 0x7fce68100630 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6ca47640 1 -- 172.21.15.7:0/1539653978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce681a1ac0 con 0x7fce68100630 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce577fe640 1 -- 172.21.15.7:0/1539653978 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce6000cd80 con 0x7fce68100630 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce577fe640 1 -- 172.21.15.7:0/1539653978 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60010430 con 0x7fce68100630 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.832+0000 7fce6ca47640 1 -- 172.21.15.7:0/1539653978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce681a1f50 con 0x7fce68100630 2024-03-23T13:17:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.833+0000 7fce6ca47640 1 -- 172.21.15.7:0/1539653978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce6810c040 con 0x7fce68100630 2024-03-23T13:17:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.834+0000 7fce577fe640 1 -- 172.21.15.7:0/1539653978 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fce60003680 con 0x7fce68100630 2024-03-23T13:17:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.835+0000 7fce577fe640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce500a2150 0x7fce500a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.835+0000 7fce65d9b640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce500a2150 0x7fce500a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.835+0000 7fce577fe640 1 -- 172.21.15.7:0/1539653978 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fce60014070 con 0x7fce68100630 2024-03-23T13:17:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.835+0000 7fce65d9b640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce500a2150 0x7fce500a4610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009230 tx=0x7fce5c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:49.839+0000 7fce577fe640 1 -- 172.21.15.7:0/1539653978 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fce60097030 con 0x7fce68100630 2024-03-23T13:17:49.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:49 smithi130 ceph-mon[36225]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s wr, 19 op/s 2024-03-23T13:17:49.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:49 smithi130 ceph-mon[36225]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:49.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:49 smithi130 ceph-mon[36225]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-23T13:17:50.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.060+0000 7fce6ca47640 1 -- 172.21.15.7:0/1539653978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce681a21a0 con 0x7fce68100630 2024-03-23T13:17:50.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.060+0000 7fce577fe640 1 -- 172.21.15.7:0/1539653978 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11917 (secure 0 0 0) 0x7fce6009a6f0 con 0x7fce68100630 2024-03-23T13:17:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:50.060 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T13:17:40.597309+0000","modified":"2024-03-23T13:17:48.259350+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24475,"mds_3":24470,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6841/3415570481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3415570481},{"type":"v1","addr":"172.21.15.7:6841","nonce":3415570481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/791500055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":791500055},{"type":"v1","addr":"172.21.15.130:6839","nonce":791500055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24470,24463,24475,24503,24496]},"id":1}]} 2024-03-23T13:17:50.060 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-23T13:17:50.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.062+0000 7fce6ca47640 1 -- 172.21.15.7:0/1539653978 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce500a2150 msgr2=0x7fce500a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.062+0000 7fce6ca47640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce500a2150 0x7fce500a4610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009230 tx=0x7fce5c005fd0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.062+0000 7fce6ca47640 1 -- 172.21.15.7:0/1539653978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68100630 msgr2=0x7fce681999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.062+0000 7fce6ca47640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68100630 0x7fce681999c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9c0 tx=0x7fce6000ee80 comp rx=0 tx=0).stop 2024-03-23T13:17:50.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.062+0000 7fce6ca47640 1 -- 172.21.15.7:0/1539653978 shutdown_connections 2024-03-23T13:17:50.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.062+0000 7fce6ca47640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce500a2150 0x7fce500a4610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.062+0000 7fce6ca47640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce681021e0 0x7fce681a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.062+0000 7fce6ca47640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce68101820 0x7fce68199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.062+0000 7fce6ca47640 1 --2- 172.21.15.7:0/1539653978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68100630 0x7fce681999c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.063+0000 7fce6ca47640 1 -- 172.21.15.7:0/1539653978 >> 172.21.15.7:0/1539653978 conn(0x7fce680fbba0 msgr2=0x7fce680fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.063+0000 7fce6ca47640 1 -- 172.21.15.7:0/1539653978 shutdown_connections 2024-03-23T13:17:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.063+0000 7fce6ca47640 1 -- 172.21.15.7:0/1539653978 wait complete. 2024-03-23T13:17:50.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:17:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.176+0000 7f8ce9525640 1 -- 172.21.15.7:0/977697552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce4101820 msgr2=0x7f8ce4101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.176+0000 7f8ce9525640 1 --2- 172.21.15.7:0/977697552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce4101820 0x7f8ce4101ca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009900 tx=0x7f8cd402bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.176+0000 7f8ce9525640 1 -- 172.21.15.7:0/977697552 shutdown_connections 2024-03-23T13:17:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.176+0000 7f8ce9525640 1 --2- 172.21.15.7:0/977697552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce41021e0 0x7f8ce4108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.176+0000 7f8ce9525640 1 --2- 172.21.15.7:0/977697552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce4101820 0x7f8ce4101ca0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.176+0000 7f8ce9525640 1 --2- 172.21.15.7:0/977697552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce4100630 0x7f8ce4100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.176+0000 7f8ce9525640 1 -- 172.21.15.7:0/977697552 >> 172.21.15.7:0/977697552 conn(0x7f8ce40fbba0 msgr2=0x7f8ce40fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.176+0000 7f8ce9525640 1 -- 172.21.15.7:0/977697552 shutdown_connections 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.176+0000 7f8ce9525640 1 -- 172.21.15.7:0/977697552 wait complete. 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce9525640 1 Processor -- start 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce9525640 1 -- start start 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce9525640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce4100630 0x7f8ce41977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce9525640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce4101820 0x7f8ce4197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce9525640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41021e0 0x7f8ce419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce9525640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce419f2b0 con 0x7f8ce41021e0 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce9525640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce419f420 con 0x7f8ce4100630 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce9525640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce419f5e0 con 0x7f8ce4101820 2024-03-23T13:17:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41021e0 0x7f8ce419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41021e0 0x7f8ce419ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52666/0 (socket says 172.21.15.7:52666) 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce37fe640 1 -- 172.21.15.7:0/975334302 learned_addr learned my addr 172.21.15.7:0/975334302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce2ffd640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce4100630 0x7f8ce41977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce27fc640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce4101820 0x7f8ce4197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce37fe640 1 -- 172.21.15.7:0/975334302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce4101820 msgr2=0x7f8ce4197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce37fe640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce4101820 0x7f8ce4197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce37fe640 1 -- 172.21.15.7:0/975334302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce4100630 msgr2=0x7f8ce41977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce37fe640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce4100630 0x7f8ce41977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.177+0000 7f8ce37fe640 1 -- 172.21.15.7:0/975334302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd4009560 con 0x7f8ce41021e0 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.178+0000 7f8ce37fe640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41021e0 0x7f8ce419ed70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00eb10 tx=0x7f8cdc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.178+0000 7f8cdbfff640 1 -- 172.21.15.7:0/975334302 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc004b20 con 0x7f8ce41021e0 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.178+0000 7f8cdbfff640 1 -- 172.21.15.7:0/975334302 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cdc004c80 con 0x7f8ce41021e0 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.178+0000 7f8ce9525640 1 -- 172.21.15.7:0/975334302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce419f8c0 con 0x7f8ce41021e0 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.178+0000 7f8cdbfff640 1 -- 172.21.15.7:0/975334302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc010650 con 0x7f8ce41021e0 2024-03-23T13:17:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.178+0000 7f8ce9525640 1 -- 172.21.15.7:0/975334302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce419fda0 con 0x7f8ce41021e0 2024-03-23T13:17:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.179+0000 7f8ce9525640 1 -- 172.21.15.7:0/975334302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce4107950 con 0x7f8ce41021e0 2024-03-23T13:17:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.180+0000 7f8cdbfff640 1 -- 172.21.15.7:0/975334302 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f8cdc005020 con 0x7f8ce41021e0 2024-03-23T13:17:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.180+0000 7f8cdbfff640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd00a2150 0x7f8cd00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.180+0000 7f8ce2ffd640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd00a2150 0x7f8cd00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.180+0000 7f8cdbfff640 1 -- 172.21.15.7:0/975334302 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f8cdc014070 con 0x7f8ce41021e0 2024-03-23T13:17:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.181+0000 7f8ce2ffd640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd00a2150 0x7f8cd00a4610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4101680 tx=0x7f8ccc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.185+0000 7f8cdbfff640 1 -- 172.21.15.7:0/975334302 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f8cdc0971f0 con 0x7f8ce41021e0 2024-03-23T13:17:50.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.370+0000 7f8ce9525640 1 -- 172.21.15.7:0/975334302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ce410d140 con 0x7f8ce41021e0 2024-03-23T13:17:50.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.371+0000 7f8cdbfff640 1 -- 172.21.15.7:0/975334302 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22124 (secure 0 0 0) 0x7f8cdc09a8b0 con 0x7f8ce41021e0 2024-03-23T13:17:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:50.371 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:17:42.226684+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:17:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 -- 172.21.15.7:0/975334302 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd00a2150 msgr2=0x7f8cd00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd00a2150 0x7f8cd00a4610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4101680 tx=0x7f8ccc005f90 comp rx=0 tx=0).stop 2024-03-23T13:17:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 -- 172.21.15.7:0/975334302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41021e0 msgr2=0x7f8ce419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41021e0 0x7f8ce419ed70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00eb10 tx=0x7f8cdc00c2d0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 -- 172.21.15.7:0/975334302 shutdown_connections 2024-03-23T13:17:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8cd00a2150 0x7f8cd00a4610 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41021e0 0x7f8ce419ed70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce4101820 0x7f8ce4197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 --2- 172.21.15.7:0/975334302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ce4100630 0x7f8ce41977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 -- 172.21.15.7:0/975334302 >> 172.21.15.7:0/975334302 conn(0x7f8ce40fbba0 msgr2=0x7f8ce4104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 -- 172.21.15.7:0/975334302 shutdown_connections 2024-03-23T13:17:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.374+0000 7f8ce9525640 1 -- 172.21.15.7:0/975334302 wait complete. 2024-03-23T13:17:50.385 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T13:17:50.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T13:17:50.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.488+0000 7f70e3bd0640 1 -- 172.21.15.7:0/3022374319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc101bf0 msgr2=0x7f70dc102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.488+0000 7f70e3bd0640 1 --2- 172.21.15.7:0/3022374319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc101bf0 0x7f70dc102010 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009860 tx=0x7f70d802bb70 comp rx=0 tx=0).stop 2024-03-23T13:17:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.488+0000 7f70e3bd0640 1 -- 172.21.15.7:0/3022374319 shutdown_connections 2024-03-23T13:17:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.488+0000 7f70e3bd0640 1 --2- 172.21.15.7:0/3022374319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70dc1012e0 0x7f70dc108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.488+0000 7f70e3bd0640 1 --2- 172.21.15.7:0/3022374319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70dc100920 0x7f70dc100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.488+0000 7f70e3bd0640 1 --2- 172.21.15.7:0/3022374319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc101bf0 0x7f70dc102010 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.488+0000 7f70e3bd0640 1 -- 172.21.15.7:0/3022374319 >> 172.21.15.7:0/3022374319 conn(0x7f70dc0fb350 msgr2=0x7f70dc0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.488+0000 7f70e3bd0640 1 -- 172.21.15.7:0/3022374319 shutdown_connections 2024-03-23T13:17:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.488+0000 7f70e3bd0640 1 -- 172.21.15.7:0/3022374319 wait complete. 2024-03-23T13:17:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.488+0000 7f70e3bd0640 1 Processor -- start 2024-03-23T13:17:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e3bd0640 1 -- start start 2024-03-23T13:17:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e3bd0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70dc100920 0x7f70dc197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e3bd0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc1012e0 0x7f70dc197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e3bd0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70dc101bf0 0x7f70dc19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e3bd0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc19f340 con 0x7f70dc101bf0 2024-03-23T13:17:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e3bd0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc19f4b0 con 0x7f70dc100920 2024-03-23T13:17:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e3bd0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc19f670 con 0x7f70dc1012e0 2024-03-23T13:17:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e216d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70dc101bf0 0x7f70dc19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e196c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70dc100920 0x7f70dc197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e216d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70dc101bf0 0x7f70dc19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52678/0 (socket says 172.21.15.7:52678) 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e216d640 1 -- 172.21.15.7:0/26422609 learned_addr learned my addr 172.21.15.7:0/26422609 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e116b640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc1012e0 0x7f70dc197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e216d640 1 -- 172.21.15.7:0/26422609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc1012e0 msgr2=0x7f70dc197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e216d640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc1012e0 0x7f70dc197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e216d640 1 -- 172.21.15.7:0/26422609 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70dc100920 msgr2=0x7f70dc197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e216d640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70dc100920 0x7f70dc197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e216d640 1 -- 172.21.15.7:0/26422609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d4009900 con 0x7f70dc101bf0 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e196c640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70dc100920 0x7f70dc197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e216d640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70dc101bf0 0x7f70dc19ee00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f70d400efc0 tx=0x7f70d400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70ceffd640 1 -- 172.21.15.7:0/26422609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4003970 con 0x7f70dc101bf0 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e3bd0640 1 -- 172.21.15.7:0/26422609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d8009490 con 0x7f70dc101bf0 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70ceffd640 1 -- 172.21.15.7:0/26422609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d4004020 con 0x7f70dc101bf0 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70ceffd640 1 -- 172.21.15.7:0/26422609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d400b9b0 con 0x7f70dc101bf0 2024-03-23T13:17:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.489+0000 7f70e3bd0640 1 -- 172.21.15.7:0/26422609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc19fc80 con 0x7f70dc101bf0 2024-03-23T13:17:50.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.490+0000 7f70e3bd0640 1 -- 172.21.15.7:0/26422609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70dc107b30 con 0x7f70dc101bf0 2024-03-23T13:17:50.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.491+0000 7f70ceffd640 1 -- 172.21.15.7:0/26422609 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f70d400a040 con 0x7f70dc101bf0 2024-03-23T13:17:50.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.492+0000 7f70ceffd640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f70c80a2150 0x7f70c80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.492+0000 7f70e196c640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f70c80a2150 0x7f70c80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.492+0000 7f70ceffd640 1 -- 172.21.15.7:0/26422609 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f70d40cdc90 con 0x7f70dc101bf0 2024-03-23T13:17:50.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.492+0000 7f70e196c640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f70c80a2150 0x7f70c80a4610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009830 tx=0x7f70d8034c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.496+0000 7f70ceffd640 1 -- 172.21.15.7:0/26422609 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f70d4096cb0 con 0x7f70dc101bf0 2024-03-23T13:17:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.673+0000 7f70e3bd0640 1 -- 172.21.15.7:0/26422609 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70dc02d240 con 0x7f70c80a2150 2024-03-23T13:17:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.681+0000 7f70ceffd640 1 -- 172.21.15.7:0/26422609 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258877 (secure 0 0 0) 0x7f70dc02d240 con 0x7f70c80a2150 2024-03-23T13:17:50.684 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 -- 172.21.15.7:0/26422609 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f70c80a2150 msgr2=0x7f70c80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f70c80a2150 0x7f70c80a4610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009830 tx=0x7f70d8034c30 comp rx=0 tx=0).stop 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 -- 172.21.15.7:0/26422609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70dc101bf0 msgr2=0x7f70dc19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70dc101bf0 0x7f70dc19ee00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f70d400efc0 tx=0x7f70d400c800 comp rx=0 tx=0).stop 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 -- 172.21.15.7:0/26422609 shutdown_connections 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f70c80a2150 0x7f70c80a4610 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70dc101bf0 0x7f70dc19ee00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc1012e0 0x7f70dc197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 --2- 172.21.15.7:0/26422609 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70dc100920 0x7f70dc197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 -- 172.21.15.7:0/26422609 >> 172.21.15.7:0/26422609 conn(0x7f70dc0fb350 msgr2=0x7f70dc104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 -- 172.21.15.7:0/26422609 shutdown_connections 2024-03-23T13:17:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.684+0000 7f70e3bd0640 1 -- 172.21.15.7:0/26422609 wait complete. 2024-03-23T13:17:50.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:50 smithi007 ceph-mon[32184]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:50.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:50 smithi007 ceph-mon[32184]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:50.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:50 smithi007 ceph-mon[32184]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:50.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:50 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1539653978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:50.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:50 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/975334302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:50.695 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-23T13:17:49.925276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596783,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":332,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372780,"kb_used_data":11124,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569044,"statfs":{"total":1151940427776,"available":1151558701056,"internally_reserved":0,"allocated":11390976,"data_stored":7046226,"data_compressed":108357,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6415,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.698453"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622805+0000","last_change":"2024-03-23T13:17:41.250141+0000","last_active":"2024-03-23T13:17:42.622805+0000","last_peered":"2024-03-23T13:17:42.622805+0000","last_clean":"2024-03-23T13:17:42.622805+0000","last_became_active":"2024-03-23T13:17:41.249578+0000","last_became_peered":"2024-03-23T13:17:41.249578+0000","last_unstale":"2024-03-23T13:17:42.622805+0000","last_undegraded":"2024-03-23T13:17:42.622805+0000","last_fullsized":"2024-03-23T13:17:42.622805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:45.858553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585361+0000","last_change":"2024-03-23T13:17:41.251945+0000","last_active":"2024-03-23T13:17:42.585361+0000","last_peered":"2024-03-23T13:17:42.585361+0000","last_clean":"2024-03-23T13:17:42.585361+0000","last_became_active":"2024-03-23T13:17:41.251603+0000","last_became_peered":"2024-03-23T13:17:41.251603+0000","last_unstale":"2024-03-23T13:17:42.585361+0000","last_undegraded":"2024-03-23T13:17:42.585361+0000","last_fullsized":"2024-03-23T13:17:42.585361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:08:55.160949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585470+0000","last_change":"2024-03-23T13:17:41.254293+0000","last_active":"2024-03-23T13:17:42.585470+0000","last_peered":"2024-03-23T13:17:42.585470+0000","last_clean":"2024-03-23T13:17:42.585470+0000","last_became_active":"2024-03-23T13:17:41.253981+0000","last_became_peered":"2024-03-23T13:17:41.253981+0000","last_unstale":"2024-03-23T13:17:42.585470+0000","last_undegraded":"2024-03-23T13:17:42.585470+0000","last_fullsized":"2024-03-23T13:17:42.585470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:23:17.445169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621592+0000","last_change":"2024-03-23T13:17:41.252125+0000","last_active":"2024-03-23T13:17:42.621592+0000","last_peered":"2024-03-23T13:17:42.621592+0000","last_clean":"2024-03-23T13:17:42.621592+0000","last_became_active":"2024-03-23T13:17:41.251819+0000","last_became_peered":"2024-03-23T13:17:41.251819+0000","last_unstale":"2024-03-23T13:17:42.621592+0000","last_undegraded":"2024-03-23T13:17:42.621592+0000","last_fullsized":"2024-03-23T13:17:42.621592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:31.559237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622592+0000","last_change":"2024-03-23T13:17:41.244653+0000","last_active":"2024-03-23T13:17:42.622592+0000","last_peered":"2024-03-23T13:17:42.622592+0000","last_clean":"2024-03-23T13:17:42.622592+0000","last_became_active":"2024-03-23T13:17:41.244322+0000","last_became_peered":"2024-03-23T13:17:41.244322+0000","last_unstale":"2024-03-23T13:17:42.622592+0000","last_undegraded":"2024-03-23T13:17:42.622592+0000","last_fullsized":"2024-03-23T13:17:42.622592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:55:19.419296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.540436+0000","last_change":"2024-03-23T13:17:41.248093+0000","last_active":"2024-03-23T13:17:42.540436+0000","last_peered":"2024-03-23T13:17:42.540436+0000","last_clean":"2024-03-23T13:17:42.540436+0000","last_became_active":"2024-03-23T13:17:41.247788+0000","last_became_peered":"2024-03-23T13:17:41.247788+0000","last_unstale":"2024-03-23T13:17:42.540436+0000","last_undegraded":"2024-03-23T13:17:42.540436+0000","last_fullsized":"2024-03-23T13:17:42.540436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:53:11.888143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.808816+0000","last_change":"2024-03-23T13:17:41.250721+0000","last_active":"2024-03-23T13:17:42.808816+0000","last_peered":"2024-03-23T13:17:42.808816+0000","last_clean":"2024-03-23T13:17:42.808816+0000","last_became_active":"2024-03-23T13:17:41.250403+0000","last_became_peered":"2024-03-23T13:17:41.250403+0000","last_unstale":"2024-03-23T13:17:42.808816+0000","last_undegraded":"2024-03-23T13:17:42.808816+0000","last_fullsized":"2024-03-23T13:17:42.808816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:51:54.657066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621150+0000","last_change":"2024-03-23T13:17:41.252899+0000","last_active":"2024-03-23T13:17:42.621150+0000","last_peered":"2024-03-23T13:17:42.621150+0000","last_clean":"2024-03-23T13:17:42.621150+0000","last_became_active":"2024-03-23T13:17:41.252584+0000","last_became_peered":"2024-03-23T13:17:41.252584+0000","last_unstale":"2024-03-23T13:17:42.621150+0000","last_undegraded":"2024-03-23T13:17:42.621150+0000","last_fullsized":"2024-03-23T13:17:42.621150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:29:33.752127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.236444+0000","last_change":"2024-03-23T13:17:41.253829+0000","last_active":"2024-03-23T13:17:42.236444+0000","last_peered":"2024-03-23T13:17:42.236444+0000","last_clean":"2024-03-23T13:17:42.236444+0000","last_became_active":"2024-03-23T13:17:41.253520+0000","last_became_peered":"2024-03-23T13:17:41.253520+0000","last_unstale":"2024-03-23T13:17:42.236444+0000","last_undegraded":"2024-03-23T13:17:42.236444+0000","last_fullsized":"2024-03-23T13:17:42.236444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:27:40.159325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.540720+0000","last_change":"2024-03-23T13:17:41.249801+0000","last_active":"2024-03-23T13:17:42.540720+0000","last_peered":"2024-03-23T13:17:42.540720+0000","last_clean":"2024-03-23T13:17:42.540720+0000","last_became_active":"2024-03-23T13:17:41.249491+0000","last_became_peered":"2024-03-23T13:17:41.249491+0000","last_unstale":"2024-03-23T13:17:42.540720+0000","last_undegraded":"2024-03-23T13:17:42.540720+0000","last_fullsized":"2024-03-23T13:17:42.540720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:37:02.132652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809134+0000","last_change":"2024-03-23T13:17:41.257957+0000","last_active":"2024-03-23T13:17:42.809134+0000","last_peered":"2024-03-23T13:17:42.809134+0000","last_clean":"2024-03-23T13:17:42.809134+0000","last_became_active":"2024-03-23T13:17:41.257651+0000","last_became_peered":"2024-03-23T13:17:41.257651+0000","last_unstale":"2024-03-23T13:17:42.809134+0000","last_undegraded":"2024-03-23T13:17:42.809134+0000","last_fullsized":"2024-03-23T13:17:42.809134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:42:33.120763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621551+0000","last_change":"2024-03-23T13:17:41.254935+0000","last_active":"2024-03-23T13:17:42.621551+0000","last_peered":"2024-03-23T13:17:42.621551+0000","last_clean":"2024-03-23T13:17:42.621551+0000","last_became_active":"2024-03-23T13:17:41.254633+0000","last_became_peered":"2024-03-23T13:17:41.254633+0000","last_unstale":"2024-03-23T13:17:42.621551+0000","last_undegraded":"2024-03-23T13:17:42.621551+0000","last_fullsized":"2024-03-23T13:17:42.621551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:23:18.345884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234430+0000","last_change":"2024-03-23T13:17:41.233447+0000","last_active":"2024-03-23T13:17:42.234430+0000","last_peered":"2024-03-23T13:17:42.234430+0000","last_clean":"2024-03-23T13:17:42.234430+0000","last_became_active":"2024-03-23T13:17:41.233143+0000","last_became_peered":"2024-03-23T13:17:41.233143+0000","last_unstale":"2024-03-23T13:17:42.234430+0000","last_undegraded":"2024-03-23T13:17:42.234430+0000","last_fullsized":"2024-03-23T13:17:42.234430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:16:53.677024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585280+0000","last_change":"2024-03-23T13:17:41.242668+0000","last_active":"2024-03-23T13:17:42.585280+0000","last_peered":"2024-03-23T13:17:42.585280+0000","last_clean":"2024-03-23T13:17:42.585280+0000","last_became_active":"2024-03-23T13:17:41.242321+0000","last_became_peered":"2024-03-23T13:17:41.242321+0000","last_unstale":"2024-03-23T13:17:42.585280+0000","last_undegraded":"2024-03-23T13:17:42.585280+0000","last_fullsized":"2024-03-23T13:17:42.585280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:17.783814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585034+0000","last_change":"2024-03-23T13:17:41.249476+0000","last_active":"2024-03-23T13:17:42.585034+0000","last_peered":"2024-03-23T13:17:42.585034+0000","last_clean":"2024-03-23T13:17:42.585034+0000","last_became_active":"2024-03-23T13:17:41.248467+0000","last_became_peered":"2024-03-23T13:17:41.248467+0000","last_unstale":"2024-03-23T13:17:42.585034+0000","last_undegraded":"2024-03-23T13:17:42.585034+0000","last_fullsized":"2024-03-23T13:17:42.585034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:21:40.537751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.235869+0000","last_change":"2024-03-23T13:17:41.253679+0000","last_active":"2024-03-23T13:17:42.235869+0000","last_peered":"2024-03-23T13:17:42.235869+0000","last_clean":"2024-03-23T13:17:42.235869+0000","last_became_active":"2024-03-23T13:17:41.253371+0000","last_became_peered":"2024-03-23T13:17:41.253371+0000","last_unstale":"2024-03-23T13:17:42.235869+0000","last_undegraded":"2024-03-23T13:17:42.235869+0000","last_fullsized":"2024-03-23T13:17:42.235869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:16:08.397446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622915+0000","last_change":"2024-03-23T13:17:41.249856+0000","last_active":"2024-03-23T13:17:42.622915+0000","last_peered":"2024-03-23T13:17:42.622915+0000","last_clean":"2024-03-23T13:17:42.622915+0000","last_became_active":"2024-03-23T13:17:41.249163+0000","last_became_peered":"2024-03-23T13:17:41.249163+0000","last_unstale":"2024-03-23T13:17:42.622915+0000","last_undegraded":"2024-03-23T13:17:42.622915+0000","last_fullsized":"2024-03-23T13:17:42.622915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:14.184859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621611+0000","last_change":"2024-03-23T13:17:41.250048+0000","last_active":"2024-03-23T13:17:42.621611+0000","last_peered":"2024-03-23T13:17:42.621611+0000","last_clean":"2024-03-23T13:17:42.621611+0000","last_became_active":"2024-03-23T13:17:41.249491+0000","last_became_peered":"2024-03-23T13:17:41.249491+0000","last_unstale":"2024-03-23T13:17:42.621611+0000","last_undegraded":"2024-03-23T13:17:42.621611+0000","last_fullsized":"2024-03-23T13:17:42.621611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:55:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234981+0000","last_change":"2024-03-23T13:17:41.256615+0000","last_active":"2024-03-23T13:17:42.234981+0000","last_peered":"2024-03-23T13:17:42.234981+0000","last_clean":"2024-03-23T13:17:42.234981+0000","last_became_active":"2024-03-23T13:17:41.256287+0000","last_became_peered":"2024-03-23T13:17:41.256287+0000","last_unstale":"2024-03-23T13:17:42.234981+0000","last_undegraded":"2024-03-23T13:17:42.234981+0000","last_fullsized":"2024-03-23T13:17:42.234981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:31.686699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:47.580655+0000","last_change":"2024-03-23T13:17:40.231464+0000","last_active":"2024-03-23T13:17:47.580655+0000","last_peered":"2024-03-23T13:17:47.580655+0000","last_clean":"2024-03-23T13:17:47.580655+0000","last_became_active":"2024-03-23T13:17:40.230711+0000","last_became_peered":"2024-03-23T13:17:40.230711+0000","last_unstale":"2024-03-23T13:17:47.580655+0000","last_undegraded":"2024-03-23T13:17:47.580655+0000","last_fullsized":"2024-03-23T13:17:47.580655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:17.926187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621778+0000","last_change":"2024-03-23T13:17:41.251449+0000","last_active":"2024-03-23T13:17:42.621778+0000","last_peered":"2024-03-23T13:17:42.621778+0000","last_clean":"2024-03-23T13:17:42.621778+0000","last_became_active":"2024-03-23T13:17:41.251077+0000","last_became_peered":"2024-03-23T13:17:41.251077+0000","last_unstale":"2024-03-23T13:17:42.621778+0000","last_undegraded":"2024-03-23T13:17:42.621778+0000","last_fullsized":"2024-03-23T13:17:42.621778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:09.939244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621941+0000","last_change":"2024-03-23T13:17:40.233263+0000","last_active":"2024-03-23T13:17:42.621941+0000","last_peered":"2024-03-23T13:17:42.621941+0000","last_clean":"2024-03-23T13:17:42.621941+0000","last_became_active":"2024-03-23T13:17:40.232686+0000","last_became_peered":"2024-03-23T13:17:40.232686+0000","last_unstale":"2024-03-23T13:17:42.621941+0000","last_undegraded":"2024-03-23T13:17:42.621941+0000","last_fullsized":"2024-03-23T13:17:42.621941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:26.671088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585479+0000","last_change":"2024-03-23T13:17:41.250532+0000","last_active":"2024-03-23T13:17:42.585479+0000","last_peered":"2024-03-23T13:17:42.585479+0000","last_clean":"2024-03-23T13:17:42.585479+0000","last_became_active":"2024-03-23T13:17:41.250230+0000","last_became_peered":"2024-03-23T13:17:41.250230+0000","last_unstale":"2024-03-23T13:17:42.585479+0000","last_undegraded":"2024-03-23T13:17:42.585479+0000","last_fullsized":"2024-03-23T13:17:42.585479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:13:27.406827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:46.573456+0000","last_change":"2024-03-23T13:17:40.237333+0000","last_active":"2024-03-23T13:17:46.573456+0000","last_peered":"2024-03-23T13:17:46.573456+0000","last_clean":"2024-03-23T13:17:46.573456+0000","last_became_active":"2024-03-23T13:17:40.237007+0000","last_became_peered":"2024-03-23T13:17:40.237007+0000","last_unstale":"2024-03-23T13:17:46.573456+0000","last_undegraded":"2024-03-23T13:17:46.573456+0000","last_fullsized":"2024-03-23T13:17:46.573456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:24:21.611645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.236122+0000","last_change":"2024-03-23T13:17:41.247529+0000","last_active":"2024-03-23T13:17:42.236122+0000","last_peered":"2024-03-23T13:17:42.236122+0000","last_clean":"2024-03-23T13:17:42.236122+0000","last_became_active":"2024-03-23T13:17:41.247008+0000","last_became_peered":"2024-03-23T13:17:41.247008+0000","last_unstale":"2024-03-23T13:17:42.236122+0000","last_undegraded":"2024-03-23T13:17:42.236122+0000","last_fullsized":"2024-03-23T13:17:42.236122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:22.015602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622635+0000","last_change":"2024-03-23T13:17:40.233544+0000","last_active":"2024-03-23T13:17:42.622635+0000","last_peered":"2024-03-23T13:17:42.622635+0000","last_clean":"2024-03-23T13:17:42.622635+0000","last_became_active":"2024-03-23T13:17:40.233222+0000","last_became_peered":"2024-03-23T13:17:40.233222+0000","last_unstale":"2024-03-23T13:17:42.622635+0000","last_undegraded":"2024-03-23T13:17:42.622635+0000","last_fullsized":"2024-03-23T13:17:42.622635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:15:32.147020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622230+0000","last_change":"2024-03-23T13:17:41.245643+0000","last_active":"2024-03-23T13:17:42.622230+0000","last_peered":"2024-03-23T13:17:42.622230+0000","last_clean":"2024-03-23T13:17:42.622230+0000","last_became_active":"2024-03-23T13:17:41.245295+0000","last_became_peered":"2024-03-23T13:17:41.245295+0000","last_unstale":"2024-03-23T13:17:42.622230+0000","last_undegraded":"2024-03-23T13:17:42.622230+0000","last_fullsized":"2024-03-23T13:17:42.622230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:27:26.480225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.556770+0000","last_change":"2024-03-23T13:17:40.236473+0000","last_active":"2024-03-23T13:17:44.556770+0000","last_peered":"2024-03-23T13:17:44.556770+0000","last_clean":"2024-03-23T13:17:44.556770+0000","last_became_active":"2024-03-23T13:17:40.236114+0000","last_became_peered":"2024-03-23T13:17:40.236114+0000","last_unstale":"2024-03-23T13:17:44.556770+0000","last_undegraded":"2024-03-23T13:17:44.556770+0000","last_fullsized":"2024-03-23T13:17:44.556770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:37:33.949014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622075+0000","last_change":"2024-03-23T13:17:41.241904+0000","last_active":"2024-03-23T13:17:42.622075+0000","last_peered":"2024-03-23T13:17:42.622075+0000","last_clean":"2024-03-23T13:17:42.622075+0000","last_became_active":"2024-03-23T13:17:41.241605+0000","last_became_peered":"2024-03-23T13:17:41.241605+0000","last_unstale":"2024-03-23T13:17:42.622075+0000","last_undegraded":"2024-03-23T13:17:42.622075+0000","last_fullsized":"2024-03-23T13:17:42.622075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:05:43.782714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:46.574802+0000","last_change":"2024-03-23T13:17:40.232399+0000","last_active":"2024-03-23T13:17:46.574802+0000","last_peered":"2024-03-23T13:17:46.574802+0000","last_clean":"2024-03-23T13:17:46.574802+0000","last_became_active":"2024-03-23T13:17:40.232071+0000","last_became_peered":"2024-03-23T13:17:40.232071+0000","last_unstale":"2024-03-23T13:17:46.574802+0000","last_undegraded":"2024-03-23T13:17:46.574802+0000","last_fullsized":"2024-03-23T13:17:46.574802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:34:35.839756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621787+0000","last_change":"2024-03-23T13:17:41.246698+0000","last_active":"2024-03-23T13:17:42.621787+0000","last_peered":"2024-03-23T13:17:42.621787+0000","last_clean":"2024-03-23T13:17:42.621787+0000","last_became_active":"2024-03-23T13:17:41.246351+0000","last_became_peered":"2024-03-23T13:17:41.246351+0000","last_unstale":"2024-03-23T13:17:42.621787+0000","last_undegraded":"2024-03-23T13:17:42.621787+0000","last_fullsized":"2024-03-23T13:17:42.621787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:35:43.068833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.564115+0000","last_change":"2024-03-23T13:17:40.230550+0000","last_active":"2024-03-23T13:17:45.564115+0000","last_peered":"2024-03-23T13:17:45.564115+0000","last_clean":"2024-03-23T13:17:45.564115+0000","last_became_active":"2024-03-23T13:17:40.230215+0000","last_became_peered":"2024-03-23T13:17:40.230215+0000","last_unstale":"2024-03-23T13:17:45.564115+0000","last_undegraded":"2024-03-23T13:17:45.564115+0000","last_fullsized":"2024-03-23T13:17:45.564115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:48:52.998612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621334+0000","last_change":"2024-03-23T13:17:41.246704+0000","last_active":"2024-03-23T13:17:42.621334+0000","last_peered":"2024-03-23T13:17:42.621334+0000","last_clean":"2024-03-23T13:17:42.621334+0000","last_became_active":"2024-03-23T13:17:41.246405+0000","last_became_peered":"2024-03-23T13:17:41.246405+0000","last_unstale":"2024-03-23T13:17:42.621334+0000","last_undegraded":"2024-03-23T13:17:42.621334+0000","last_fullsized":"2024-03-23T13:17:42.621334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:23:10.404830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621730+0000","last_change":"2024-03-23T13:17:40.232444+0000","last_active":"2024-03-23T13:17:42.621730+0000","last_peered":"2024-03-23T13:17:42.621730+0000","last_clean":"2024-03-23T13:17:42.621730+0000","last_became_active":"2024-03-23T13:17:40.232142+0000","last_became_peered":"2024-03-23T13:17:40.232142+0000","last_unstale":"2024-03-23T13:17:42.621730+0000","last_undegraded":"2024-03-23T13:17:42.621730+0000","last_fullsized":"2024-03-23T13:17:42.621730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:32:58.171626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621255+0000","last_change":"2024-03-23T13:17:41.246894+0000","last_active":"2024-03-23T13:17:42.621255+0000","last_peered":"2024-03-23T13:17:42.621255+0000","last_clean":"2024-03-23T13:17:42.621255+0000","last_became_active":"2024-03-23T13:17:41.246537+0000","last_became_peered":"2024-03-23T13:17:41.246537+0000","last_unstale":"2024-03-23T13:17:42.621255+0000","last_undegraded":"2024-03-23T13:17:42.621255+0000","last_fullsized":"2024-03-23T13:17:42.621255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:58.878823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.556138+0000","last_change":"2024-03-23T13:17:40.231214+0000","last_active":"2024-03-23T13:17:44.556138+0000","last_peered":"2024-03-23T13:17:44.556138+0000","last_clean":"2024-03-23T13:17:44.556138+0000","last_became_active":"2024-03-23T13:17:40.230890+0000","last_became_peered":"2024-03-23T13:17:40.230890+0000","last_unstale":"2024-03-23T13:17:44.556138+0000","last_undegraded":"2024-03-23T13:17:44.556138+0000","last_fullsized":"2024-03-23T13:17:44.556138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:00:57.300516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.236401+0000","last_change":"2024-03-23T13:17:41.245560+0000","last_active":"2024-03-23T13:17:42.236401+0000","last_peered":"2024-03-23T13:17:42.236401+0000","last_clean":"2024-03-23T13:17:42.236401+0000","last_became_active":"2024-03-23T13:17:41.245192+0000","last_became_peered":"2024-03-23T13:17:41.245192+0000","last_unstale":"2024-03-23T13:17:42.236401+0000","last_undegraded":"2024-03-23T13:17:42.236401+0000","last_fullsized":"2024-03-23T13:17:42.236401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:07:02.080036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.555164+0000","last_change":"2024-03-23T13:17:40.232603+0000","last_active":"2024-03-23T13:17:44.555164+0000","last_peered":"2024-03-23T13:17:44.555164+0000","last_clean":"2024-03-23T13:17:44.555164+0000","last_became_active":"2024-03-23T13:17:40.231970+0000","last_became_peered":"2024-03-23T13:17:40.231970+0000","last_unstale":"2024-03-23T13:17:44.555164+0000","last_undegraded":"2024-03-23T13:17:44.555164+0000","last_fullsized":"2024-03-23T13:17:44.555164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:14:22.383047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585422+0000","last_change":"2024-03-23T13:17:41.244255+0000","last_active":"2024-03-23T13:17:42.585422+0000","last_peered":"2024-03-23T13:17:42.585422+0000","last_clean":"2024-03-23T13:17:42.585422+0000","last_became_active":"2024-03-23T13:17:41.243925+0000","last_became_peered":"2024-03-23T13:17:41.243925+0000","last_unstale":"2024-03-23T13:17:42.585422+0000","last_undegraded":"2024-03-23T13:17:42.585422+0000","last_fullsized":"2024-03-23T13:17:42.585422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:03.115703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622282+0000","last_change":"2024-03-23T13:17:40.229160+0000","last_active":"2024-03-23T13:17:42.622282+0000","last_peered":"2024-03-23T13:17:42.622282+0000","last_clean":"2024-03-23T13:17:42.622282+0000","last_became_active":"2024-03-23T13:17:40.228846+0000","last_became_peered":"2024-03-23T13:17:40.228846+0000","last_unstale":"2024-03-23T13:17:42.622282+0000","last_undegraded":"2024-03-23T13:17:42.622282+0000","last_fullsized":"2024-03-23T13:17:42.622282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:25.517124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.236322+0000","last_change":"2024-03-23T13:17:41.247364+0000","last_active":"2024-03-23T13:17:42.236322+0000","last_peered":"2024-03-23T13:17:42.236322+0000","last_clean":"2024-03-23T13:17:42.236322+0000","last_became_active":"2024-03-23T13:17:41.246671+0000","last_became_peered":"2024-03-23T13:17:41.246671+0000","last_unstale":"2024-03-23T13:17:42.236322+0000","last_undegraded":"2024-03-23T13:17:42.236322+0000","last_fullsized":"2024-03-23T13:17:42.236322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:50:15.654396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'11","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:47.581870+0000","last_change":"2024-03-23T13:17:40.229805+0000","last_active":"2024-03-23T13:17:47.581870+0000","last_peered":"2024-03-23T13:17:47.581870+0000","last_clean":"2024-03-23T13:17:47.581870+0000","last_became_active":"2024-03-23T13:17:40.229438+0000","last_became_peered":"2024-03-23T13:17:40.229438+0000","last_unstale":"2024-03-23T13:17:47.581870+0000","last_undegraded":"2024-03-23T13:17:47.581870+0000","last_fullsized":"2024-03-23T13:17:47.581870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:21:11.121141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2003,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622833+0000","last_change":"2024-03-23T13:17:41.250106+0000","last_active":"2024-03-23T13:17:42.622833+0000","last_peered":"2024-03-23T13:17:42.622833+0000","last_clean":"2024-03-23T13:17:42.622833+0000","last_became_active":"2024-03-23T13:17:41.249725+0000","last_became_peered":"2024-03-23T13:17:41.249725+0000","last_unstale":"2024-03-23T13:17:42.622833+0000","last_undegraded":"2024-03-23T13:17:42.622833+0000","last_fullsized":"2024-03-23T13:17:42.622833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:07:20.243310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.563812+0000","last_change":"2024-03-23T13:17:40.234631+0000","last_active":"2024-03-23T13:17:45.563812+0000","last_peered":"2024-03-23T13:17:45.563812+0000","last_clean":"2024-03-23T13:17:45.563812+0000","last_became_active":"2024-03-23T13:17:40.234294+0000","last_became_peered":"2024-03-23T13:17:40.234294+0000","last_unstale":"2024-03-23T13:17:45.563812+0000","last_undegraded":"2024-03-23T13:17:45.563812+0000","last_fullsized":"2024-03-23T13:17:45.563812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:57.058252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234218+0000","last_change":"2024-03-23T13:17:41.247425+0000","last_active":"2024-03-23T13:17:42.234218+0000","last_peered":"2024-03-23T13:17:42.234218+0000","last_clean":"2024-03-23T13:17:42.234218+0000","last_became_active":"2024-03-23T13:17:41.247125+0000","last_became_peered":"2024-03-23T13:17:41.247125+0000","last_unstale":"2024-03-23T13:17:42.234218+0000","last_undegraded":"2024-03-23T13:17:42.234218+0000","last_fullsized":"2024-03-23T13:17:42.234218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:50.118666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.555951+0000","last_change":"2024-03-23T13:17:40.236179+0000","last_active":"2024-03-23T13:17:44.555951+0000","last_peered":"2024-03-23T13:17:44.555951+0000","last_clean":"2024-03-23T13:17:44.555951+0000","last_became_active":"2024-03-23T13:17:40.235810+0000","last_became_peered":"2024-03-23T13:17:40.235810+0000","last_unstale":"2024-03-23T13:17:44.555951+0000","last_undegraded":"2024-03-23T13:17:44.555951+0000","last_fullsized":"2024-03-23T13:17:44.555951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:27:28.193164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809150+0000","last_change":"2024-03-23T13:17:41.252261+0000","last_active":"2024-03-23T13:17:42.809150+0000","last_peered":"2024-03-23T13:17:42.809150+0000","last_clean":"2024-03-23T13:17:42.809150+0000","last_became_active":"2024-03-23T13:17:41.251949+0000","last_became_peered":"2024-03-23T13:17:41.251949+0000","last_unstale":"2024-03-23T13:17:42.809150+0000","last_undegraded":"2024-03-23T13:17:42.809150+0000","last_fullsized":"2024-03-23T13:17:42.809150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:39.576999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.560996+0000","last_change":"2024-03-23T13:17:40.228738+0000","last_active":"2024-03-23T13:17:45.560996+0000","last_peered":"2024-03-23T13:17:45.560996+0000","last_clean":"2024-03-23T13:17:45.560996+0000","last_became_active":"2024-03-23T13:17:40.228424+0000","last_became_peered":"2024-03-23T13:17:40.228424+0000","last_unstale":"2024-03-23T13:17:45.560996+0000","last_undegraded":"2024-03-23T13:17:45.560996+0000","last_fullsized":"2024-03-23T13:17:45.560996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:51.506223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.808815+0000","last_change":"2024-03-23T13:17:41.247155+0000","last_active":"2024-03-23T13:17:42.808815+0000","last_peered":"2024-03-23T13:17:42.808815+0000","last_clean":"2024-03-23T13:17:42.808815+0000","last_became_active":"2024-03-23T13:17:41.246849+0000","last_became_peered":"2024-03-23T13:17:41.246849+0000","last_unstale":"2024-03-23T13:17:42.808815+0000","last_undegraded":"2024-03-23T13:17:42.808815+0000","last_fullsized":"2024-03-23T13:17:42.808815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:23.311456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.564467+0000","last_change":"2024-03-23T13:17:40.232420+0000","last_active":"2024-03-23T13:17:45.564467+0000","last_peered":"2024-03-23T13:17:45.564467+0000","last_clean":"2024-03-23T13:17:45.564467+0000","last_became_active":"2024-03-23T13:17:40.231940+0000","last_became_peered":"2024-03-23T13:17:40.231940+0000","last_unstale":"2024-03-23T13:17:45.564467+0000","last_undegraded":"2024-03-23T13:17:45.564467+0000","last_fullsized":"2024-03-23T13:17:45.564467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:46.300579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621609+0000","last_change":"2024-03-23T13:17:41.249717+0000","last_active":"2024-03-23T13:17:42.621609+0000","last_peered":"2024-03-23T13:17:42.621609+0000","last_clean":"2024-03-23T13:17:42.621609+0000","last_became_active":"2024-03-23T13:17:41.249286+0000","last_became_peered":"2024-03-23T13:17:41.249286+0000","last_unstale":"2024-03-23T13:17:42.621609+0000","last_undegraded":"2024-03-23T13:17:42.621609+0000","last_fullsized":"2024-03-23T13:17:42.621609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:47:19.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622212+0000","last_change":"2024-03-23T13:17:40.229172+0000","last_active":"2024-03-23T13:17:42.622212+0000","last_peered":"2024-03-23T13:17:42.622212+0000","last_clean":"2024-03-23T13:17:42.622212+0000","last_became_active":"2024-03-23T13:17:40.228858+0000","last_became_peered":"2024-03-23T13:17:40.228858+0000","last_unstale":"2024-03-23T13:17:42.622212+0000","last_undegraded":"2024-03-23T13:17:42.622212+0000","last_fullsized":"2024-03-23T13:17:42.622212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:17.929936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622306+0000","last_change":"2024-03-23T13:17:41.253041+0000","last_active":"2024-03-23T13:17:42.622306+0000","last_peered":"2024-03-23T13:17:42.622306+0000","last_clean":"2024-03-23T13:17:42.622306+0000","last_became_active":"2024-03-23T13:17:41.252739+0000","last_became_peered":"2024-03-23T13:17:41.252739+0000","last_unstale":"2024-03-23T13:17:42.622306+0000","last_undegraded":"2024-03-23T13:17:42.622306+0000","last_fullsized":"2024-03-23T13:17:42.622306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:27:52.560688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.692522+0000","last_change":"2024-03-23T13:17:40.228098+0000","last_active":"2024-03-23T13:17:42.692522+0000","last_peered":"2024-03-23T13:17:42.692522+0000","last_clean":"2024-03-23T13:17:42.692522+0000","last_became_active":"2024-03-23T13:17:40.227778+0000","last_became_peered":"2024-03-23T13:17:40.227778+0000","last_unstale":"2024-03-23T13:17:42.692522+0000","last_undegraded":"2024-03-23T13:17:42.692522+0000","last_fullsized":"2024-03-23T13:17:42.692522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:25:38.862329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.540610+0000","last_change":"2024-03-23T13:17:41.242666+0000","last_active":"2024-03-23T13:17:42.540610+0000","last_peered":"2024-03-23T13:17:42.540610+0000","last_clean":"2024-03-23T13:17:42.540610+0000","last_became_active":"2024-03-23T13:17:41.242371+0000","last_became_peered":"2024-03-23T13:17:41.242371+0000","last_unstale":"2024-03-23T13:17:42.540610+0000","last_undegraded":"2024-03-23T13:17:42.540610+0000","last_fullsized":"2024-03-23T13:17:42.540610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:40.870090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.808896+0000","last_change":"2024-03-23T13:17:40.233558+0000","last_active":"2024-03-23T13:17:42.808896+0000","last_peered":"2024-03-23T13:17:42.808896+0000","last_clean":"2024-03-23T13:17:42.808896+0000","last_became_active":"2024-03-23T13:17:40.232112+0000","last_became_peered":"2024-03-23T13:17:40.232112+0000","last_unstale":"2024-03-23T13:17:42.808896+0000","last_undegraded":"2024-03-23T13:17:42.808896+0000","last_fullsized":"2024-03-23T13:17:42.808896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:37:43.971598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.540634+0000","last_change":"2024-03-23T13:17:41.245804+0000","last_active":"2024-03-23T13:17:42.540634+0000","last_peered":"2024-03-23T13:17:42.540634+0000","last_clean":"2024-03-23T13:17:42.540634+0000","last_became_active":"2024-03-23T13:17:41.245509+0000","last_became_peered":"2024-03-23T13:17:41.245509+0000","last_unstale":"2024-03-23T13:17:42.540634+0000","last_undegraded":"2024-03-23T13:17:42.540634+0000","last_fullsized":"2024-03-23T13:17:42.540634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:39:55.735413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621823+0000","last_change":"2024-03-23T13:17:41.250553+0000","last_active":"2024-03-23T13:17:42.621823+0000","last_peered":"2024-03-23T13:17:42.621823+0000","last_clean":"2024-03-23T13:17:42.621823+0000","last_became_active":"2024-03-23T13:17:41.250140+0000","last_became_peered":"2024-03-23T13:17:41.250140+0000","last_unstale":"2024-03-23T13:17:42.621823+0000","last_undegraded":"2024-03-23T13:17:42.621823+0000","last_fullsized":"2024-03-23T13:17:42.621823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:28.088801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.563450+0000","last_change":"2024-03-23T13:17:40.239651+0000","last_active":"2024-03-23T13:17:45.563450+0000","last_peered":"2024-03-23T13:17:45.563450+0000","last_clean":"2024-03-23T13:17:45.563450+0000","last_became_active":"2024-03-23T13:17:40.239326+0000","last_became_peered":"2024-03-23T13:17:40.239326+0000","last_unstale":"2024-03-23T13:17:45.563450+0000","last_undegraded":"2024-03-23T13:17:45.563450+0000","last_fullsized":"2024-03-23T13:17:45.563450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:56:36.147882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621480+0000","last_change":"2024-03-23T13:17:41.253148+0000","last_active":"2024-03-23T13:17:42.621480+0000","last_peered":"2024-03-23T13:17:42.621480+0000","last_clean":"2024-03-23T13:17:42.621480+0000","last_became_active":"2024-03-23T13:17:41.252571+0000","last_became_peered":"2024-03-23T13:17:41.252571+0000","last_unstale":"2024-03-23T13:17:42.621480+0000","last_undegraded":"2024-03-23T13:17:42.621480+0000","last_fullsized":"2024-03-23T13:17:42.621480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:19:36.477844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.555550+0000","last_change":"2024-03-23T13:17:40.227457+0000","last_active":"2024-03-23T13:17:44.555550+0000","last_peered":"2024-03-23T13:17:44.555550+0000","last_clean":"2024-03-23T13:17:44.555550+0000","last_became_active":"2024-03-23T13:17:40.227149+0000","last_became_peered":"2024-03-23T13:17:40.227149+0000","last_unstale":"2024-03-23T13:17:44.555550+0000","last_undegraded":"2024-03-23T13:17:44.555550+0000","last_fullsized":"2024-03-23T13:17:44.555550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:33:20.606023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":337,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.808730+0000","last_change":"2024-03-23T13:16:31.257961+0000","last_active":"2024-03-23T13:17:42.808730+0000","last_peered":"2024-03-23T13:17:42.808730+0000","last_clean":"2024-03-23T13:17:42.808730+0000","last_became_active":"2024-03-23T13:16:31.257444+0000","last_became_peered":"2024-03-23T13:16:31.257444+0000","last_unstale":"2024-03-23T13:17:42.808730+0000","last_undegraded":"2024-03-23T13:17:42.808730+0000","last_fullsized":"2024-03-23T13:17:42.808730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:13:08.847194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:13:08.847194+0000","last_clean_scrub_stamp":"2024-03-23T13:13:08.847194+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:35:07.006029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234126+0000","last_change":"2024-03-23T13:17:41.249443+0000","last_active":"2024-03-23T13:17:42.234126+0000","last_peered":"2024-03-23T13:17:42.234126+0000","last_clean":"2024-03-23T13:17:42.234126+0000","last_became_active":"2024-03-23T13:17:41.248960+0000","last_became_peered":"2024-03-23T13:17:41.248960+0000","last_unstale":"2024-03-23T13:17:42.234126+0000","last_undegraded":"2024-03-23T13:17:42.234126+0000","last_fullsized":"2024-03-23T13:17:42.234126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:35:36.819271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.808951+0000","last_change":"2024-03-23T13:17:40.230268+0000","last_active":"2024-03-23T13:17:42.808951+0000","last_peered":"2024-03-23T13:17:42.808951+0000","last_clean":"2024-03-23T13:17:42.808951+0000","last_became_active":"2024-03-23T13:17:40.229955+0000","last_became_peered":"2024-03-23T13:17:40.229955+0000","last_unstale":"2024-03-23T13:17:42.808951+0000","last_undegraded":"2024-03-23T13:17:42.808951+0000","last_fullsized":"2024-03-23T13:17:42.808951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:38.977844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:46.574934+0000","last_change":"2024-03-23T13:17:40.235259+0000","last_active":"2024-03-23T13:17:46.574934+0000","last_peered":"2024-03-23T13:17:46.574934+0000","last_clean":"2024-03-23T13:17:46.574934+0000","last_became_active":"2024-03-23T13:17:40.234941+0000","last_became_peered":"2024-03-23T13:17:40.234941+0000","last_unstale":"2024-03-23T13:17:46.574934+0000","last_undegraded":"2024-03-23T13:17:46.574934+0000","last_fullsized":"2024-03-23T13:17:46.574934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:31:24.799836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234441+0000","last_change":"2024-03-23T13:17:41.248807+0000","last_active":"2024-03-23T13:17:42.234441+0000","last_peered":"2024-03-23T13:17:42.234441+0000","last_clean":"2024-03-23T13:17:42.234441+0000","last_became_active":"2024-03-23T13:17:41.248401+0000","last_became_peered":"2024-03-23T13:17:41.248401+0000","last_unstale":"2024-03-23T13:17:42.234441+0000","last_undegraded":"2024-03-23T13:17:42.234441+0000","last_fullsized":"2024-03-23T13:17:42.234441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:13:30.533996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:46.575596+0000","last_change":"2024-03-23T13:17:40.233323+0000","last_active":"2024-03-23T13:17:46.575596+0000","last_peered":"2024-03-23T13:17:46.575596+0000","last_clean":"2024-03-23T13:17:46.575596+0000","last_became_active":"2024-03-23T13:17:40.233031+0000","last_became_peered":"2024-03-23T13:17:40.233031+0000","last_unstale":"2024-03-23T13:17:46.575596+0000","last_undegraded":"2024-03-23T13:17:46.575596+0000","last_fullsized":"2024-03-23T13:17:46.575596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:55:35.870633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621803+0000","last_change":"2024-03-23T13:17:41.243662+0000","last_active":"2024-03-23T13:17:42.621803+0000","last_peered":"2024-03-23T13:17:42.621803+0000","last_clean":"2024-03-23T13:17:42.621803+0000","last_became_active":"2024-03-23T13:17:41.243310+0000","last_became_peered":"2024-03-23T13:17:41.243310+0000","last_unstale":"2024-03-23T13:17:42.621803+0000","last_undegraded":"2024-03-23T13:17:42.621803+0000","last_fullsized":"2024-03-23T13:17:42.621803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:25:06.277185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.553499+0000","last_change":"2024-03-23T13:17:40.226698+0000","last_active":"2024-03-23T13:17:44.553499+0000","last_peered":"2024-03-23T13:17:44.553499+0000","last_clean":"2024-03-23T13:17:44.553499+0000","last_became_active":"2024-03-23T13:17:40.226305+0000","last_became_peered":"2024-03-23T13:17:40.226305+0000","last_unstale":"2024-03-23T13:17:44.553499+0000","last_undegraded":"2024-03-23T13:17:44.553499+0000","last_fullsized":"2024-03-23T13:17:44.553499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:06:19.529125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585342+0000","last_change":"2024-03-23T13:17:41.247685+0000","last_active":"2024-03-23T13:17:42.585342+0000","last_peered":"2024-03-23T13:17:42.585342+0000","last_clean":"2024-03-23T13:17:42.585342+0000","last_became_active":"2024-03-23T13:17:41.247389+0000","last_became_peered":"2024-03-23T13:17:41.247389+0000","last_unstale":"2024-03-23T13:17:42.585342+0000","last_undegraded":"2024-03-23T13:17:42.585342+0000","last_fullsized":"2024-03-23T13:17:42.585342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:53.258831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.555358+0000","last_change":"2024-03-23T13:17:40.228860+0000","last_active":"2024-03-23T13:17:44.555358+0000","last_peered":"2024-03-23T13:17:44.555358+0000","last_clean":"2024-03-23T13:17:44.555358+0000","last_became_active":"2024-03-23T13:17:40.228535+0000","last_became_peered":"2024-03-23T13:17:40.228535+0000","last_unstale":"2024-03-23T13:17:44.555358+0000","last_undegraded":"2024-03-23T13:17:44.555358+0000","last_fullsized":"2024-03-23T13:17:44.555358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:05:33.739369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621759+0000","last_change":"2024-03-23T13:17:41.239461+0000","last_active":"2024-03-23T13:17:42.621759+0000","last_peered":"2024-03-23T13:17:42.621759+0000","last_clean":"2024-03-23T13:17:42.621759+0000","last_became_active":"2024-03-23T13:17:41.239137+0000","last_became_peered":"2024-03-23T13:17:41.239137+0000","last_unstale":"2024-03-23T13:17:42.621759+0000","last_undegraded":"2024-03-23T13:17:42.621759+0000","last_fullsized":"2024-03-23T13:17:42.621759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:06:43.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.554711+0000","last_change":"2024-03-23T13:17:40.237796+0000","last_active":"2024-03-23T13:17:44.554711+0000","last_peered":"2024-03-23T13:17:44.554711+0000","last_clean":"2024-03-23T13:17:44.554711+0000","last_became_active":"2024-03-23T13:17:40.236596+0000","last_became_peered":"2024-03-23T13:17:40.236596+0000","last_unstale":"2024-03-23T13:17:44.554711+0000","last_undegraded":"2024-03-23T13:17:44.554711+0000","last_fullsized":"2024-03-23T13:17:44.554711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:22.767459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621439+0000","last_change":"2024-03-23T13:17:41.250379+0000","last_active":"2024-03-23T13:17:42.621439+0000","last_peered":"2024-03-23T13:17:42.621439+0000","last_clean":"2024-03-23T13:17:42.621439+0000","last_became_active":"2024-03-23T13:17:41.244850+0000","last_became_peered":"2024-03-23T13:17:41.244850+0000","last_unstale":"2024-03-23T13:17:42.621439+0000","last_undegraded":"2024-03-23T13:17:42.621439+0000","last_fullsized":"2024-03-23T13:17:42.621439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:34:42.741865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.551310+0000","last_change":"2024-03-23T13:17:40.234334+0000","last_active":"2024-03-23T13:17:45.551310+0000","last_peered":"2024-03-23T13:17:45.551310+0000","last_clean":"2024-03-23T13:17:45.551310+0000","last_became_active":"2024-03-23T13:17:40.233953+0000","last_became_peered":"2024-03-23T13:17:40.233953+0000","last_unstale":"2024-03-23T13:17:45.551310+0000","last_undegraded":"2024-03-23T13:17:45.551310+0000","last_fullsized":"2024-03-23T13:17:45.551310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:53:48.238091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621770+0000","last_change":"2024-03-23T13:17:41.247960+0000","last_active":"2024-03-23T13:17:42.621770+0000","last_peered":"2024-03-23T13:17:42.621770+0000","last_clean":"2024-03-23T13:17:42.621770+0000","last_became_active":"2024-03-23T13:17:41.247459+0000","last_became_peered":"2024-03-23T13:17:41.247459+0000","last_unstale":"2024-03-23T13:17:42.621770+0000","last_undegraded":"2024-03-23T13:17:42.621770+0000","last_fullsized":"2024-03-23T13:17:42.621770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:01.345486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.556504+0000","last_change":"2024-03-23T13:17:40.237228+0000","last_active":"2024-03-23T13:17:44.556504+0000","last_peered":"2024-03-23T13:17:44.556504+0000","last_clean":"2024-03-23T13:17:44.556504+0000","last_became_active":"2024-03-23T13:17:40.236843+0000","last_became_peered":"2024-03-23T13:17:40.236843+0000","last_unstale":"2024-03-23T13:17:44.556504+0000","last_undegraded":"2024-03-23T13:17:44.556504+0000","last_fullsized":"2024-03-23T13:17:44.556504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:31:53.315272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809091+0000","last_change":"2024-03-23T13:17:41.256459+0000","last_active":"2024-03-23T13:17:42.809091+0000","last_peered":"2024-03-23T13:17:42.809091+0000","last_clean":"2024-03-23T13:17:42.809091+0000","last_became_active":"2024-03-23T13:17:41.256168+0000","last_became_peered":"2024-03-23T13:17:41.256168+0000","last_unstale":"2024-03-23T13:17:42.809091+0000","last_undegraded":"2024-03-23T13:17:42.809091+0000","last_fullsized":"2024-03-23T13:17:42.809091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:50.488371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:46.573487+0000","last_change":"2024-03-23T13:17:40.224068+0000","last_active":"2024-03-23T13:17:46.573487+0000","last_peered":"2024-03-23T13:17:46.573487+0000","last_clean":"2024-03-23T13:17:46.573487+0000","last_became_active":"2024-03-23T13:17:40.223744+0000","last_became_peered":"2024-03-23T13:17:40.223744+0000","last_unstale":"2024-03-23T13:17:46.573487+0000","last_undegraded":"2024-03-23T13:17:46.573487+0000","last_fullsized":"2024-03-23T13:17:46.573487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:08:56.187695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809207+0000","last_change":"2024-03-23T13:17:41.257665+0000","last_active":"2024-03-23T13:17:42.809207+0000","last_peered":"2024-03-23T13:17:42.809207+0000","last_clean":"2024-03-23T13:17:42.809207+0000","last_became_active":"2024-03-23T13:17:41.256943+0000","last_became_peered":"2024-03-23T13:17:41.256943+0000","last_unstale":"2024-03-23T13:17:42.809207+0000","last_undegraded":"2024-03-23T13:17:42.809207+0000","last_fullsized":"2024-03-23T13:17:42.809207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:12:21.896800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622031+0000","last_change":"2024-03-23T13:17:40.236231+0000","last_active":"2024-03-23T13:17:42.622031+0000","last_peered":"2024-03-23T13:17:42.622031+0000","last_clean":"2024-03-23T13:17:42.622031+0000","last_became_active":"2024-03-23T13:17:40.235880+0000","last_became_peered":"2024-03-23T13:17:40.235880+0000","last_unstale":"2024-03-23T13:17:42.622031+0000","last_undegraded":"2024-03-23T13:17:42.622031+0000","last_fullsized":"2024-03-23T13:17:42.622031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:38.400890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809066+0000","last_change":"2024-03-23T13:17:41.252326+0000","last_active":"2024-03-23T13:17:42.809066+0000","last_peered":"2024-03-23T13:17:42.809066+0000","last_clean":"2024-03-23T13:17:42.809066+0000","last_became_active":"2024-03-23T13:17:41.252023+0000","last_became_peered":"2024-03-23T13:17:41.252023+0000","last_unstale":"2024-03-23T13:17:42.809066+0000","last_undegraded":"2024-03-23T13:17:42.809066+0000","last_fullsized":"2024-03-23T13:17:42.809066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:18:51.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234383+0000","last_change":"2024-03-23T13:17:40.239647+0000","last_active":"2024-03-23T13:17:42.234383+0000","last_peered":"2024-03-23T13:17:42.234383+0000","last_clean":"2024-03-23T13:17:42.234383+0000","last_became_active":"2024-03-23T13:17:40.239318+0000","last_became_peered":"2024-03-23T13:17:40.239318+0000","last_unstale":"2024-03-23T13:17:42.234383+0000","last_undegraded":"2024-03-23T13:17:42.234383+0000","last_fullsized":"2024-03-23T13:17:42.234383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:56:03.354003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809154+0000","last_change":"2024-03-23T13:17:41.255446+0000","last_active":"2024-03-23T13:17:42.809154+0000","last_peered":"2024-03-23T13:17:42.809154+0000","last_clean":"2024-03-23T13:17:42.809154+0000","last_became_active":"2024-03-23T13:17:41.255135+0000","last_became_peered":"2024-03-23T13:17:41.255135+0000","last_unstale":"2024-03-23T13:17:42.809154+0000","last_undegraded":"2024-03-23T13:17:42.809154+0000","last_fullsized":"2024-03-23T13:17:42.809154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:47.639958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621468+0000","last_change":"2024-03-23T13:17:41.245392+0000","last_active":"2024-03-23T13:17:42.621468+0000","last_peered":"2024-03-23T13:17:42.621468+0000","last_clean":"2024-03-23T13:17:42.621468+0000","last_became_active":"2024-03-23T13:17:41.244439+0000","last_became_peered":"2024-03-23T13:17:41.244439+0000","last_unstale":"2024-03-23T13:17:42.621468+0000","last_undegraded":"2024-03-23T13:17:42.621468+0000","last_fullsized":"2024-03-23T13:17:42.621468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:01:50.028867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234631+0000","last_change":"2024-03-23T13:17:41.253214+0000","last_active":"2024-03-23T13:17:42.234631+0000","last_peered":"2024-03-23T13:17:42.234631+0000","last_clean":"2024-03-23T13:17:42.234631+0000","last_became_active":"2024-03-23T13:17:41.252892+0000","last_became_peered":"2024-03-23T13:17:41.252892+0000","last_unstale":"2024-03-23T13:17:42.234631+0000","last_undegraded":"2024-03-23T13:17:42.234631+0000","last_fullsized":"2024-03-23T13:17:42.234631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:24:15.475541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234582+0000","last_change":"2024-03-23T13:17:41.247561+0000","last_active":"2024-03-23T13:17:42.234582+0000","last_peered":"2024-03-23T13:17:42.234582+0000","last_clean":"2024-03-23T13:17:42.234582+0000","last_became_active":"2024-03-23T13:17:41.247238+0000","last_became_peered":"2024-03-23T13:17:41.247238+0000","last_unstale":"2024-03-23T13:17:42.234582+0000","last_undegraded":"2024-03-23T13:17:42.234582+0000","last_fullsized":"2024-03-23T13:17:42.234582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:37:19.301021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585201+0000","last_change":"2024-03-23T13:17:41.241607+0000","last_active":"2024-03-23T13:17:42.585201+0000","last_peered":"2024-03-23T13:17:42.585201+0000","last_clean":"2024-03-23T13:17:42.585201+0000","last_became_active":"2024-03-23T13:17:41.241285+0000","last_became_peered":"2024-03-23T13:17:41.241285+0000","last_unstale":"2024-03-23T13:17:42.585201+0000","last_undegraded":"2024-03-23T13:17:42.585201+0000","last_fullsized":"2024-03-23T13:17:42.585201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:34:35.800087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234420+0000","last_change":"2024-03-23T13:17:41.245423+0000","last_active":"2024-03-23T13:17:42.234420+0000","last_peered":"2024-03-23T13:17:42.234420+0000","last_clean":"2024-03-23T13:17:42.234420+0000","last_became_active":"2024-03-23T13:17:41.245051+0000","last_became_peered":"2024-03-23T13:17:41.245051+0000","last_unstale":"2024-03-23T13:17:42.234420+0000","last_undegraded":"2024-03-23T13:17:42.234420+0000","last_fullsized":"2024-03-23T13:17:42.234420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:10:23.191491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622049+0000","last_change":"2024-03-23T13:17:41.241371+0000","last_active":"2024-03-23T13:17:42.622049+0000","last_peered":"2024-03-23T13:17:42.622049+0000","last_clean":"2024-03-23T13:17:42.622049+0000","last_became_active":"2024-03-23T13:17:41.240970+0000","last_became_peered":"2024-03-23T13:17:41.240970+0000","last_unstale":"2024-03-23T13:17:42.622049+0000","last_undegraded":"2024-03-23T13:17:42.622049+0000","last_fullsized":"2024-03-23T13:17:42.622049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:14.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.235098+0000","last_change":"2024-03-23T13:17:41.248578+0000","last_active":"2024-03-23T13:17:42.235098+0000","last_peered":"2024-03-23T13:17:42.235098+0000","last_clean":"2024-03-23T13:17:42.235098+0000","last_became_active":"2024-03-23T13:17:41.248264+0000","last_became_peered":"2024-03-23T13:17:41.248264+0000","last_unstale":"2024-03-23T13:17:42.235098+0000","last_undegraded":"2024-03-23T13:17:42.235098+0000","last_fullsized":"2024-03-23T13:17:42.235098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:42.937816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621682+0000","last_change":"2024-03-23T13:17:41.246138+0000","last_active":"2024-03-23T13:17:42.621682+0000","last_peered":"2024-03-23T13:17:42.621682+0000","last_clean":"2024-03-23T13:17:42.621682+0000","last_became_active":"2024-03-23T13:17:41.245596+0000","last_became_peered":"2024-03-23T13:17:41.245596+0000","last_unstale":"2024-03-23T13:17:42.621682+0000","last_undegraded":"2024-03-23T13:17:42.621682+0000","last_fullsized":"2024-03-23T13:17:42.621682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:51:43.524257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621437+0000","last_change":"2024-03-23T13:17:41.246819+0000","last_active":"2024-03-23T13:17:42.621437+0000","last_peered":"2024-03-23T13:17:42.621437+0000","last_clean":"2024-03-23T13:17:42.621437+0000","last_became_active":"2024-03-23T13:17:41.246389+0000","last_became_peered":"2024-03-23T13:17:41.246389+0000","last_unstale":"2024-03-23T13:17:42.621437+0000","last_undegraded":"2024-03-23T13:17:42.621437+0000","last_fullsized":"2024-03-23T13:17:42.621437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:40.759387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621230+0000","last_change":"2024-03-23T13:17:41.254185+0000","last_active":"2024-03-23T13:17:42.621230+0000","last_peered":"2024-03-23T13:17:42.621230+0000","last_clean":"2024-03-23T13:17:42.621230+0000","last_became_active":"2024-03-23T13:17:41.253828+0000","last_became_peered":"2024-03-23T13:17:41.253828+0000","last_unstale":"2024-03-23T13:17:42.621230+0000","last_undegraded":"2024-03-23T13:17:42.621230+0000","last_fullsized":"2024-03-23T13:17:42.621230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:27.826109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.236101+0000","last_change":"2024-03-23T13:17:41.257669+0000","last_active":"2024-03-23T13:17:42.236101+0000","last_peered":"2024-03-23T13:17:42.236101+0000","last_clean":"2024-03-23T13:17:42.236101+0000","last_became_active":"2024-03-23T13:17:41.257341+0000","last_became_peered":"2024-03-23T13:17:41.257341+0000","last_unstale":"2024-03-23T13:17:42.236101+0000","last_undegraded":"2024-03-23T13:17:42.236101+0000","last_fullsized":"2024-03-23T13:17:42.236101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:58:08.777906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234551+0000","last_change":"2024-03-23T13:17:41.249078+0000","last_active":"2024-03-23T13:17:42.234551+0000","last_peered":"2024-03-23T13:17:42.234551+0000","last_clean":"2024-03-23T13:17:42.234551+0000","last_became_active":"2024-03-23T13:17:41.247804+0000","last_became_peered":"2024-03-23T13:17:41.247804+0000","last_unstale":"2024-03-23T13:17:42.234551+0000","last_undegraded":"2024-03-23T13:17:42.234551+0000","last_fullsized":"2024-03-23T13:17:42.234551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:20:19.514109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621764+0000","last_change":"2024-03-23T13:17:41.242707+0000","last_active":"2024-03-23T13:17:42.621764+0000","last_peered":"2024-03-23T13:17:42.621764+0000","last_clean":"2024-03-23T13:17:42.621764+0000","last_became_active":"2024-03-23T13:17:41.242367+0000","last_became_peered":"2024-03-23T13:17:41.242367+0000","last_unstale":"2024-03-23T13:17:42.621764+0000","last_undegraded":"2024-03-23T13:17:42.621764+0000","last_fullsized":"2024-03-23T13:17:42.621764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:14:30.695624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249559,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6415,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":20307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1032633,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":439257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":438873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332129,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":1029085,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":438225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":440329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":440313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":441097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":438329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920816,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":438779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":1029979,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34482,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34482,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T13:17:50.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T13:17:50.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 -- 172.21.15.7:0/332902045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6164101830 msgr2=0x7f6164101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 --2- 172.21.15.7:0/332902045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6164101830 0x7f6164101cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6158009900 tx=0x7f615802bc30 comp rx=0 tx=0).stop 2024-03-23T13:17:50.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 -- 172.21.15.7:0/332902045 shutdown_connections 2024-03-23T13:17:50.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 --2- 172.21.15.7:0/332902045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61641021f0 0x7f6164108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 --2- 172.21.15.7:0/332902045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6164101830 0x7f6164101cb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 --2- 172.21.15.7:0/332902045 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6164100640 0x7f6164100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 -- 172.21.15.7:0/332902045 >> 172.21.15.7:0/332902045 conn(0x7f61640fbbf0 msgr2=0x7f61640fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 -- 172.21.15.7:0/332902045 shutdown_connections 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 -- 172.21.15.7:0/332902045 wait complete. 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 Processor -- start 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.839+0000 7f616c31e640 1 -- start start 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616c31e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6164100640 0x7f616406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616c31e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6164101830 0x7f616419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616c31e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61641021f0 0x7f616419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616c31e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f616419f2e0 con 0x7f6164100640 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616c31e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f616419f450 con 0x7f6164101830 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616c31e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f616419f610 con 0x7f61641021f0 2024-03-23T13:17:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616a0ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6164100640 0x7f616406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616a0ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6164100640 0x7f616406a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52690/0 (socket says 172.21.15.7:52690) 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616a0ba640 1 -- 172.21.15.7:0/892402367 learned_addr learned my addr 172.21.15.7:0/892402367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f61698b9640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6164101830 0x7f616419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616a8bb640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61641021f0 0x7f616419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616a0ba640 1 -- 172.21.15.7:0/892402367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61641021f0 msgr2=0x7f616419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616a0ba640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61641021f0 0x7f616419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616a0ba640 1 -- 172.21.15.7:0/892402367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6164101830 msgr2=0x7f616419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616a0ba640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6164101830 0x7f616419c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616a0ba640 1 -- 172.21.15.7:0/892402367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6158009560 con 0x7f6164100640 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616a0ba640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6164100640 0x7f616406a8f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f615c00ebc0 tx=0x7f615c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f61637fe640 1 -- 172.21.15.7:0/892402367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c003b70 con 0x7f6164100640 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f61637fe640 1 -- 172.21.15.7:0/892402367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f615c00eec0 con 0x7f6164100640 2024-03-23T13:17:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616c31e640 1 -- 172.21.15.7:0/892402367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616419f890 con 0x7f6164100640 2024-03-23T13:17:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f61637fe640 1 -- 172.21.15.7:0/892402367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c019800 con 0x7f6164100640 2024-03-23T13:17:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.840+0000 7f616c31e640 1 -- 172.21.15.7:0/892402367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616419fd70 con 0x7f6164100640 2024-03-23T13:17:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.841+0000 7f616c31e640 1 -- 172.21.15.7:0/892402367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6164107960 con 0x7f6164100640 2024-03-23T13:17:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.842+0000 7f61637fe640 1 -- 172.21.15.7:0/892402367 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f615c019960 con 0x7f6164100640 2024-03-23T13:17:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.843+0000 7f61637fe640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f614c0a21d0 0x7f614c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.843+0000 7f61698b9640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f614c0a21d0 0x7f614c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.843+0000 7f61637fe640 1 -- 172.21.15.7:0/892402367 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f615c014070 con 0x7f6164100640 2024-03-23T13:17:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.843+0000 7f61698b9640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f614c0a21d0 0x7f614c0a4690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f615802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:50.848+0000 7f61637fe640 1 -- 172.21.15.7:0/892402367 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f615c0974e0 con 0x7f6164100640 2024-03-23T13:17:50.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:50 smithi130 ceph-mon[36225]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:50.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:50 smithi130 ceph-mon[36225]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:50.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:50 smithi130 ceph-mon[36225]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:50.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:50 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1539653978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:50.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:50 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/975334302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:51.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.026+0000 7f616c31e640 1 -- 172.21.15.7:0/892402367 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f616402d270 con 0x7f614c0a21d0 2024-03-23T13:17:51.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.033+0000 7f61637fe640 1 -- 172.21.15.7:0/892402367 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258877 (secure 0 0 0) 0x7f616402d270 con 0x7f614c0a21d0 2024-03-23T13:17:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:51.035 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-23T13:17:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 -- 172.21.15.7:0/892402367 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f614c0a21d0 msgr2=0x7f614c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f614c0a21d0 0x7f614c0a4690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f615802c7e0 comp rx=0 tx=0).stop 2024-03-23T13:17:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 -- 172.21.15.7:0/892402367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6164100640 msgr2=0x7f616406a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6164100640 0x7f616406a8f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f615c00ebc0 tx=0x7f615c00c6a0 comp rx=0 tx=0).stop 2024-03-23T13:17:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 -- 172.21.15.7:0/892402367 shutdown_connections 2024-03-23T13:17:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f614c0a21d0 0x7f614c0a4690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61641021f0 0x7f616419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6164101830 0x7f616419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 --2- 172.21.15.7:0/892402367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6164100640 0x7f616406a8f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 -- 172.21.15.7:0/892402367 >> 172.21.15.7:0/892402367 conn(0x7f61640fbbf0 msgr2=0x7f6164104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:51.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 -- 172.21.15.7:0/892402367 shutdown_connections 2024-03-23T13:17:51.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:51.036+0000 7f616c31e640 1 -- 172.21.15.7:0/892402367 wait complete. 2024-03-23T13:17:51.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:50 smithi155 ceph-mon[36157]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:51.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:50 smithi155 ceph-mon[36157]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:51.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:50 smithi155 ceph-mon[36157]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:17:51.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:50 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1539653978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:17:51.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:50 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/975334302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:51.046 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-23T13:17:49.925276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596783,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":332,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372780,"kb_used_data":11124,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569044,"statfs":{"total":1151940427776,"available":1151558701056,"internally_reserved":0,"allocated":11390976,"data_stored":7046226,"data_compressed":108357,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6415,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.698453"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622805+0000","last_change":"2024-03-23T13:17:41.250141+0000","last_active":"2024-03-23T13:17:42.622805+0000","last_peered":"2024-03-23T13:17:42.622805+0000","last_clean":"2024-03-23T13:17:42.622805+0000","last_became_active":"2024-03-23T13:17:41.249578+0000","last_became_peered":"2024-03-23T13:17:41.249578+0000","last_unstale":"2024-03-23T13:17:42.622805+0000","last_undegraded":"2024-03-23T13:17:42.622805+0000","last_fullsized":"2024-03-23T13:17:42.622805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:45.858553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585361+0000","last_change":"2024-03-23T13:17:41.251945+0000","last_active":"2024-03-23T13:17:42.585361+0000","last_peered":"2024-03-23T13:17:42.585361+0000","last_clean":"2024-03-23T13:17:42.585361+0000","last_became_active":"2024-03-23T13:17:41.251603+0000","last_became_peered":"2024-03-23T13:17:41.251603+0000","last_unstale":"2024-03-23T13:17:42.585361+0000","last_undegraded":"2024-03-23T13:17:42.585361+0000","last_fullsized":"2024-03-23T13:17:42.585361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:08:55.160949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585470+0000","last_change":"2024-03-23T13:17:41.254293+0000","last_active":"2024-03-23T13:17:42.585470+0000","last_peered":"2024-03-23T13:17:42.585470+0000","last_clean":"2024-03-23T13:17:42.585470+0000","last_became_active":"2024-03-23T13:17:41.253981+0000","last_became_peered":"2024-03-23T13:17:41.253981+0000","last_unstale":"2024-03-23T13:17:42.585470+0000","last_undegraded":"2024-03-23T13:17:42.585470+0000","last_fullsized":"2024-03-23T13:17:42.585470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:23:17.445169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621592+0000","last_change":"2024-03-23T13:17:41.252125+0000","last_active":"2024-03-23T13:17:42.621592+0000","last_peered":"2024-03-23T13:17:42.621592+0000","last_clean":"2024-03-23T13:17:42.621592+0000","last_became_active":"2024-03-23T13:17:41.251819+0000","last_became_peered":"2024-03-23T13:17:41.251819+0000","last_unstale":"2024-03-23T13:17:42.621592+0000","last_undegraded":"2024-03-23T13:17:42.621592+0000","last_fullsized":"2024-03-23T13:17:42.621592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:31.559237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622592+0000","last_change":"2024-03-23T13:17:41.244653+0000","last_active":"2024-03-23T13:17:42.622592+0000","last_peered":"2024-03-23T13:17:42.622592+0000","last_clean":"2024-03-23T13:17:42.622592+0000","last_became_active":"2024-03-23T13:17:41.244322+0000","last_became_peered":"2024-03-23T13:17:41.244322+0000","last_unstale":"2024-03-23T13:17:42.622592+0000","last_undegraded":"2024-03-23T13:17:42.622592+0000","last_fullsized":"2024-03-23T13:17:42.622592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:55:19.419296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.540436+0000","last_change":"2024-03-23T13:17:41.248093+0000","last_active":"2024-03-23T13:17:42.540436+0000","last_peered":"2024-03-23T13:17:42.540436+0000","last_clean":"2024-03-23T13:17:42.540436+0000","last_became_active":"2024-03-23T13:17:41.247788+0000","last_became_peered":"2024-03-23T13:17:41.247788+0000","last_unstale":"2024-03-23T13:17:42.540436+0000","last_undegraded":"2024-03-23T13:17:42.540436+0000","last_fullsized":"2024-03-23T13:17:42.540436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:53:11.888143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.808816+0000","last_change":"2024-03-23T13:17:41.250721+0000","last_active":"2024-03-23T13:17:42.808816+0000","last_peered":"2024-03-23T13:17:42.808816+0000","last_clean":"2024-03-23T13:17:42.808816+0000","last_became_active":"2024-03-23T13:17:41.250403+0000","last_became_peered":"2024-03-23T13:17:41.250403+0000","last_unstale":"2024-03-23T13:17:42.808816+0000","last_undegraded":"2024-03-23T13:17:42.808816+0000","last_fullsized":"2024-03-23T13:17:42.808816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:51:54.657066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621150+0000","last_change":"2024-03-23T13:17:41.252899+0000","last_active":"2024-03-23T13:17:42.621150+0000","last_peered":"2024-03-23T13:17:42.621150+0000","last_clean":"2024-03-23T13:17:42.621150+0000","last_became_active":"2024-03-23T13:17:41.252584+0000","last_became_peered":"2024-03-23T13:17:41.252584+0000","last_unstale":"2024-03-23T13:17:42.621150+0000","last_undegraded":"2024-03-23T13:17:42.621150+0000","last_fullsized":"2024-03-23T13:17:42.621150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:29:33.752127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.236444+0000","last_change":"2024-03-23T13:17:41.253829+0000","last_active":"2024-03-23T13:17:42.236444+0000","last_peered":"2024-03-23T13:17:42.236444+0000","last_clean":"2024-03-23T13:17:42.236444+0000","last_became_active":"2024-03-23T13:17:41.253520+0000","last_became_peered":"2024-03-23T13:17:41.253520+0000","last_unstale":"2024-03-23T13:17:42.236444+0000","last_undegraded":"2024-03-23T13:17:42.236444+0000","last_fullsized":"2024-03-23T13:17:42.236444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:27:40.159325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.540720+0000","last_change":"2024-03-23T13:17:41.249801+0000","last_active":"2024-03-23T13:17:42.540720+0000","last_peered":"2024-03-23T13:17:42.540720+0000","last_clean":"2024-03-23T13:17:42.540720+0000","last_became_active":"2024-03-23T13:17:41.249491+0000","last_became_peered":"2024-03-23T13:17:41.249491+0000","last_unstale":"2024-03-23T13:17:42.540720+0000","last_undegraded":"2024-03-23T13:17:42.540720+0000","last_fullsized":"2024-03-23T13:17:42.540720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:37:02.132652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809134+0000","last_change":"2024-03-23T13:17:41.257957+0000","last_active":"2024-03-23T13:17:42.809134+0000","last_peered":"2024-03-23T13:17:42.809134+0000","last_clean":"2024-03-23T13:17:42.809134+0000","last_became_active":"2024-03-23T13:17:41.257651+0000","last_became_peered":"2024-03-23T13:17:41.257651+0000","last_unstale":"2024-03-23T13:17:42.809134+0000","last_undegraded":"2024-03-23T13:17:42.809134+0000","last_fullsized":"2024-03-23T13:17:42.809134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:42:33.120763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621551+0000","last_change":"2024-03-23T13:17:41.254935+0000","last_active":"2024-03-23T13:17:42.621551+0000","last_peered":"2024-03-23T13:17:42.621551+0000","last_clean":"2024-03-23T13:17:42.621551+0000","last_became_active":"2024-03-23T13:17:41.254633+0000","last_became_peered":"2024-03-23T13:17:41.254633+0000","last_unstale":"2024-03-23T13:17:42.621551+0000","last_undegraded":"2024-03-23T13:17:42.621551+0000","last_fullsized":"2024-03-23T13:17:42.621551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:23:18.345884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234430+0000","last_change":"2024-03-23T13:17:41.233447+0000","last_active":"2024-03-23T13:17:42.234430+0000","last_peered":"2024-03-23T13:17:42.234430+0000","last_clean":"2024-03-23T13:17:42.234430+0000","last_became_active":"2024-03-23T13:17:41.233143+0000","last_became_peered":"2024-03-23T13:17:41.233143+0000","last_unstale":"2024-03-23T13:17:42.234430+0000","last_undegraded":"2024-03-23T13:17:42.234430+0000","last_fullsized":"2024-03-23T13:17:42.234430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:16:53.677024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585280+0000","last_change":"2024-03-23T13:17:41.242668+0000","last_active":"2024-03-23T13:17:42.585280+0000","last_peered":"2024-03-23T13:17:42.585280+0000","last_clean":"2024-03-23T13:17:42.585280+0000","last_became_active":"2024-03-23T13:17:41.242321+0000","last_became_peered":"2024-03-23T13:17:41.242321+0000","last_unstale":"2024-03-23T13:17:42.585280+0000","last_undegraded":"2024-03-23T13:17:42.585280+0000","last_fullsized":"2024-03-23T13:17:42.585280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:17.783814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585034+0000","last_change":"2024-03-23T13:17:41.249476+0000","last_active":"2024-03-23T13:17:42.585034+0000","last_peered":"2024-03-23T13:17:42.585034+0000","last_clean":"2024-03-23T13:17:42.585034+0000","last_became_active":"2024-03-23T13:17:41.248467+0000","last_became_peered":"2024-03-23T13:17:41.248467+0000","last_unstale":"2024-03-23T13:17:42.585034+0000","last_undegraded":"2024-03-23T13:17:42.585034+0000","last_fullsized":"2024-03-23T13:17:42.585034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:21:40.537751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.235869+0000","last_change":"2024-03-23T13:17:41.253679+0000","last_active":"2024-03-23T13:17:42.235869+0000","last_peered":"2024-03-23T13:17:42.235869+0000","last_clean":"2024-03-23T13:17:42.235869+0000","last_became_active":"2024-03-23T13:17:41.253371+0000","last_became_peered":"2024-03-23T13:17:41.253371+0000","last_unstale":"2024-03-23T13:17:42.235869+0000","last_undegraded":"2024-03-23T13:17:42.235869+0000","last_fullsized":"2024-03-23T13:17:42.235869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:16:08.397446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622915+0000","last_change":"2024-03-23T13:17:41.249856+0000","last_active":"2024-03-23T13:17:42.622915+0000","last_peered":"2024-03-23T13:17:42.622915+0000","last_clean":"2024-03-23T13:17:42.622915+0000","last_became_active":"2024-03-23T13:17:41.249163+0000","last_became_peered":"2024-03-23T13:17:41.249163+0000","last_unstale":"2024-03-23T13:17:42.622915+0000","last_undegraded":"2024-03-23T13:17:42.622915+0000","last_fullsized":"2024-03-23T13:17:42.622915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:14.184859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621611+0000","last_change":"2024-03-23T13:17:41.250048+0000","last_active":"2024-03-23T13:17:42.621611+0000","last_peered":"2024-03-23T13:17:42.621611+0000","last_clean":"2024-03-23T13:17:42.621611+0000","last_became_active":"2024-03-23T13:17:41.249491+0000","last_became_peered":"2024-03-23T13:17:41.249491+0000","last_unstale":"2024-03-23T13:17:42.621611+0000","last_undegraded":"2024-03-23T13:17:42.621611+0000","last_fullsized":"2024-03-23T13:17:42.621611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:55:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234981+0000","last_change":"2024-03-23T13:17:41.256615+0000","last_active":"2024-03-23T13:17:42.234981+0000","last_peered":"2024-03-23T13:17:42.234981+0000","last_clean":"2024-03-23T13:17:42.234981+0000","last_became_active":"2024-03-23T13:17:41.256287+0000","last_became_peered":"2024-03-23T13:17:41.256287+0000","last_unstale":"2024-03-23T13:17:42.234981+0000","last_undegraded":"2024-03-23T13:17:42.234981+0000","last_fullsized":"2024-03-23T13:17:42.234981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:31.686699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:47.580655+0000","last_change":"2024-03-23T13:17:40.231464+0000","last_active":"2024-03-23T13:17:47.580655+0000","last_peered":"2024-03-23T13:17:47.580655+0000","last_clean":"2024-03-23T13:17:47.580655+0000","last_became_active":"2024-03-23T13:17:40.230711+0000","last_became_peered":"2024-03-23T13:17:40.230711+0000","last_unstale":"2024-03-23T13:17:47.580655+0000","last_undegraded":"2024-03-23T13:17:47.580655+0000","last_fullsized":"2024-03-23T13:17:47.580655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:17.926187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621778+0000","last_change":"2024-03-23T13:17:41.251449+0000","last_active":"2024-03-23T13:17:42.621778+0000","last_peered":"2024-03-23T13:17:42.621778+0000","last_clean":"2024-03-23T13:17:42.621778+0000","last_became_active":"2024-03-23T13:17:41.251077+0000","last_became_peered":"2024-03-23T13:17:41.251077+0000","last_unstale":"2024-03-23T13:17:42.621778+0000","last_undegraded":"2024-03-23T13:17:42.621778+0000","last_fullsized":"2024-03-23T13:17:42.621778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:09.939244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621941+0000","last_change":"2024-03-23T13:17:40.233263+0000","last_active":"2024-03-23T13:17:42.621941+0000","last_peered":"2024-03-23T13:17:42.621941+0000","last_clean":"2024-03-23T13:17:42.621941+0000","last_became_active":"2024-03-23T13:17:40.232686+0000","last_became_peered":"2024-03-23T13:17:40.232686+0000","last_unstale":"2024-03-23T13:17:42.621941+0000","last_undegraded":"2024-03-23T13:17:42.621941+0000","last_fullsized":"2024-03-23T13:17:42.621941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:26.671088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585479+0000","last_change":"2024-03-23T13:17:41.250532+0000","last_active":"2024-03-23T13:17:42.585479+0000","last_peered":"2024-03-23T13:17:42.585479+0000","last_clean":"2024-03-23T13:17:42.585479+0000","last_became_active":"2024-03-23T13:17:41.250230+0000","last_became_peered":"2024-03-23T13:17:41.250230+0000","last_unstale":"2024-03-23T13:17:42.585479+0000","last_undegraded":"2024-03-23T13:17:42.585479+0000","last_fullsized":"2024-03-23T13:17:42.585479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:13:27.406827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:46.573456+0000","last_change":"2024-03-23T13:17:40.237333+0000","last_active":"2024-03-23T13:17:46.573456+0000","last_peered":"2024-03-23T13:17:46.573456+0000","last_clean":"2024-03-23T13:17:46.573456+0000","last_became_active":"2024-03-23T13:17:40.237007+0000","last_became_peered":"2024-03-23T13:17:40.237007+0000","last_unstale":"2024-03-23T13:17:46.573456+0000","last_undegraded":"2024-03-23T13:17:46.573456+0000","last_fullsized":"2024-03-23T13:17:46.573456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:24:21.611645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.236122+0000","last_change":"2024-03-23T13:17:41.247529+0000","last_active":"2024-03-23T13:17:42.236122+0000","last_peered":"2024-03-23T13:17:42.236122+0000","last_clean":"2024-03-23T13:17:42.236122+0000","last_became_active":"2024-03-23T13:17:41.247008+0000","last_became_peered":"2024-03-23T13:17:41.247008+0000","last_unstale":"2024-03-23T13:17:42.236122+0000","last_undegraded":"2024-03-23T13:17:42.236122+0000","last_fullsized":"2024-03-23T13:17:42.236122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:22.015602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622635+0000","last_change":"2024-03-23T13:17:40.233544+0000","last_active":"2024-03-23T13:17:42.622635+0000","last_peered":"2024-03-23T13:17:42.622635+0000","last_clean":"2024-03-23T13:17:42.622635+0000","last_became_active":"2024-03-23T13:17:40.233222+0000","last_became_peered":"2024-03-23T13:17:40.233222+0000","last_unstale":"2024-03-23T13:17:42.622635+0000","last_undegraded":"2024-03-23T13:17:42.622635+0000","last_fullsized":"2024-03-23T13:17:42.622635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:15:32.147020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622230+0000","last_change":"2024-03-23T13:17:41.245643+0000","last_active":"2024-03-23T13:17:42.622230+0000","last_peered":"2024-03-23T13:17:42.622230+0000","last_clean":"2024-03-23T13:17:42.622230+0000","last_became_active":"2024-03-23T13:17:41.245295+0000","last_became_peered":"2024-03-23T13:17:41.245295+0000","last_unstale":"2024-03-23T13:17:42.622230+0000","last_undegraded":"2024-03-23T13:17:42.622230+0000","last_fullsized":"2024-03-23T13:17:42.622230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:27:26.480225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.556770+0000","last_change":"2024-03-23T13:17:40.236473+0000","last_active":"2024-03-23T13:17:44.556770+0000","last_peered":"2024-03-23T13:17:44.556770+0000","last_clean":"2024-03-23T13:17:44.556770+0000","last_became_active":"2024-03-23T13:17:40.236114+0000","last_became_peered":"2024-03-23T13:17:40.236114+0000","last_unstale":"2024-03-23T13:17:44.556770+0000","last_undegraded":"2024-03-23T13:17:44.556770+0000","last_fullsized":"2024-03-23T13:17:44.556770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:37:33.949014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622075+0000","last_change":"2024-03-23T13:17:41.241904+0000","last_active":"2024-03-23T13:17:42.622075+0000","last_peered":"2024-03-23T13:17:42.622075+0000","last_clean":"2024-03-23T13:17:42.622075+0000","last_became_active":"2024-03-23T13:17:41.241605+0000","last_became_peered":"2024-03-23T13:17:41.241605+0000","last_unstale":"2024-03-23T13:17:42.622075+0000","last_undegraded":"2024-03-23T13:17:42.622075+0000","last_fullsized":"2024-03-23T13:17:42.622075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:05:43.782714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:46.574802+0000","last_change":"2024-03-23T13:17:40.232399+0000","last_active":"2024-03-23T13:17:46.574802+0000","last_peered":"2024-03-23T13:17:46.574802+0000","last_clean":"2024-03-23T13:17:46.574802+0000","last_became_active":"2024-03-23T13:17:40.232071+0000","last_became_peered":"2024-03-23T13:17:40.232071+0000","last_unstale":"2024-03-23T13:17:46.574802+0000","last_undegraded":"2024-03-23T13:17:46.574802+0000","last_fullsized":"2024-03-23T13:17:46.574802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:34:35.839756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621787+0000","last_change":"2024-03-23T13:17:41.246698+0000","last_active":"2024-03-23T13:17:42.621787+0000","last_peered":"2024-03-23T13:17:42.621787+0000","last_clean":"2024-03-23T13:17:42.621787+0000","last_became_active":"2024-03-23T13:17:41.246351+0000","last_became_peered":"2024-03-23T13:17:41.246351+0000","last_unstale":"2024-03-23T13:17:42.621787+0000","last_undegraded":"2024-03-23T13:17:42.621787+0000","last_fullsized":"2024-03-23T13:17:42.621787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:35:43.068833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.564115+0000","last_change":"2024-03-23T13:17:40.230550+0000","last_active":"2024-03-23T13:17:45.564115+0000","last_peered":"2024-03-23T13:17:45.564115+0000","last_clean":"2024-03-23T13:17:45.564115+0000","last_became_active":"2024-03-23T13:17:40.230215+0000","last_became_peered":"2024-03-23T13:17:40.230215+0000","last_unstale":"2024-03-23T13:17:45.564115+0000","last_undegraded":"2024-03-23T13:17:45.564115+0000","last_fullsized":"2024-03-23T13:17:45.564115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:48:52.998612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621334+0000","last_change":"2024-03-23T13:17:41.246704+0000","last_active":"2024-03-23T13:17:42.621334+0000","last_peered":"2024-03-23T13:17:42.621334+0000","last_clean":"2024-03-23T13:17:42.621334+0000","last_became_active":"2024-03-23T13:17:41.246405+0000","last_became_peered":"2024-03-23T13:17:41.246405+0000","last_unstale":"2024-03-23T13:17:42.621334+0000","last_undegraded":"2024-03-23T13:17:42.621334+0000","last_fullsized":"2024-03-23T13:17:42.621334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:23:10.404830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621730+0000","last_change":"2024-03-23T13:17:40.232444+0000","last_active":"2024-03-23T13:17:42.621730+0000","last_peered":"2024-03-23T13:17:42.621730+0000","last_clean":"2024-03-23T13:17:42.621730+0000","last_became_active":"2024-03-23T13:17:40.232142+0000","last_became_peered":"2024-03-23T13:17:40.232142+0000","last_unstale":"2024-03-23T13:17:42.621730+0000","last_undegraded":"2024-03-23T13:17:42.621730+0000","last_fullsized":"2024-03-23T13:17:42.621730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:32:58.171626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621255+0000","last_change":"2024-03-23T13:17:41.246894+0000","last_active":"2024-03-23T13:17:42.621255+0000","last_peered":"2024-03-23T13:17:42.621255+0000","last_clean":"2024-03-23T13:17:42.621255+0000","last_became_active":"2024-03-23T13:17:41.246537+0000","last_became_peered":"2024-03-23T13:17:41.246537+0000","last_unstale":"2024-03-23T13:17:42.621255+0000","last_undegraded":"2024-03-23T13:17:42.621255+0000","last_fullsized":"2024-03-23T13:17:42.621255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:58.878823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.556138+0000","last_change":"2024-03-23T13:17:40.231214+0000","last_active":"2024-03-23T13:17:44.556138+0000","last_peered":"2024-03-23T13:17:44.556138+0000","last_clean":"2024-03-23T13:17:44.556138+0000","last_became_active":"2024-03-23T13:17:40.230890+0000","last_became_peered":"2024-03-23T13:17:40.230890+0000","last_unstale":"2024-03-23T13:17:44.556138+0000","last_undegraded":"2024-03-23T13:17:44.556138+0000","last_fullsized":"2024-03-23T13:17:44.556138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:00:57.300516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.236401+0000","last_change":"2024-03-23T13:17:41.245560+0000","last_active":"2024-03-23T13:17:42.236401+0000","last_peered":"2024-03-23T13:17:42.236401+0000","last_clean":"2024-03-23T13:17:42.236401+0000","last_became_active":"2024-03-23T13:17:41.245192+0000","last_became_peered":"2024-03-23T13:17:41.245192+0000","last_unstale":"2024-03-23T13:17:42.236401+0000","last_undegraded":"2024-03-23T13:17:42.236401+0000","last_fullsized":"2024-03-23T13:17:42.236401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:07:02.080036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.555164+0000","last_change":"2024-03-23T13:17:40.232603+0000","last_active":"2024-03-23T13:17:44.555164+0000","last_peered":"2024-03-23T13:17:44.555164+0000","last_clean":"2024-03-23T13:17:44.555164+0000","last_became_active":"2024-03-23T13:17:40.231970+0000","last_became_peered":"2024-03-23T13:17:40.231970+0000","last_unstale":"2024-03-23T13:17:44.555164+0000","last_undegraded":"2024-03-23T13:17:44.555164+0000","last_fullsized":"2024-03-23T13:17:44.555164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:14:22.383047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585422+0000","last_change":"2024-03-23T13:17:41.244255+0000","last_active":"2024-03-23T13:17:42.585422+0000","last_peered":"2024-03-23T13:17:42.585422+0000","last_clean":"2024-03-23T13:17:42.585422+0000","last_became_active":"2024-03-23T13:17:41.243925+0000","last_became_peered":"2024-03-23T13:17:41.243925+0000","last_unstale":"2024-03-23T13:17:42.585422+0000","last_undegraded":"2024-03-23T13:17:42.585422+0000","last_fullsized":"2024-03-23T13:17:42.585422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:03.115703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622282+0000","last_change":"2024-03-23T13:17:40.229160+0000","last_active":"2024-03-23T13:17:42.622282+0000","last_peered":"2024-03-23T13:17:42.622282+0000","last_clean":"2024-03-23T13:17:42.622282+0000","last_became_active":"2024-03-23T13:17:40.228846+0000","last_became_peered":"2024-03-23T13:17:40.228846+0000","last_unstale":"2024-03-23T13:17:42.622282+0000","last_undegraded":"2024-03-23T13:17:42.622282+0000","last_fullsized":"2024-03-23T13:17:42.622282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:25.517124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.236322+0000","last_change":"2024-03-23T13:17:41.247364+0000","last_active":"2024-03-23T13:17:42.236322+0000","last_peered":"2024-03-23T13:17:42.236322+0000","last_clean":"2024-03-23T13:17:42.236322+0000","last_became_active":"2024-03-23T13:17:41.246671+0000","last_became_peered":"2024-03-23T13:17:41.246671+0000","last_unstale":"2024-03-23T13:17:42.236322+0000","last_undegraded":"2024-03-23T13:17:42.236322+0000","last_fullsized":"2024-03-23T13:17:42.236322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:50:15.654396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'11","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:47.581870+0000","last_change":"2024-03-23T13:17:40.229805+0000","last_active":"2024-03-23T13:17:47.581870+0000","last_peered":"2024-03-23T13:17:47.581870+0000","last_clean":"2024-03-23T13:17:47.581870+0000","last_became_active":"2024-03-23T13:17:40.229438+0000","last_became_peered":"2024-03-23T13:17:40.229438+0000","last_unstale":"2024-03-23T13:17:47.581870+0000","last_undegraded":"2024-03-23T13:17:47.581870+0000","last_fullsized":"2024-03-23T13:17:47.581870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:21:11.121141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2003,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622833+0000","last_change":"2024-03-23T13:17:41.250106+0000","last_active":"2024-03-23T13:17:42.622833+0000","last_peered":"2024-03-23T13:17:42.622833+0000","last_clean":"2024-03-23T13:17:42.622833+0000","last_became_active":"2024-03-23T13:17:41.249725+0000","last_became_peered":"2024-03-23T13:17:41.249725+0000","last_unstale":"2024-03-23T13:17:42.622833+0000","last_undegraded":"2024-03-23T13:17:42.622833+0000","last_fullsized":"2024-03-23T13:17:42.622833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:07:20.243310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.563812+0000","last_change":"2024-03-23T13:17:40.234631+0000","last_active":"2024-03-23T13:17:45.563812+0000","last_peered":"2024-03-23T13:17:45.563812+0000","last_clean":"2024-03-23T13:17:45.563812+0000","last_became_active":"2024-03-23T13:17:40.234294+0000","last_became_peered":"2024-03-23T13:17:40.234294+0000","last_unstale":"2024-03-23T13:17:45.563812+0000","last_undegraded":"2024-03-23T13:17:45.563812+0000","last_fullsized":"2024-03-23T13:17:45.563812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:57.058252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234218+0000","last_change":"2024-03-23T13:17:41.247425+0000","last_active":"2024-03-23T13:17:42.234218+0000","last_peered":"2024-03-23T13:17:42.234218+0000","last_clean":"2024-03-23T13:17:42.234218+0000","last_became_active":"2024-03-23T13:17:41.247125+0000","last_became_peered":"2024-03-23T13:17:41.247125+0000","last_unstale":"2024-03-23T13:17:42.234218+0000","last_undegraded":"2024-03-23T13:17:42.234218+0000","last_fullsized":"2024-03-23T13:17:42.234218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:50.118666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.555951+0000","last_change":"2024-03-23T13:17:40.236179+0000","last_active":"2024-03-23T13:17:44.555951+0000","last_peered":"2024-03-23T13:17:44.555951+0000","last_clean":"2024-03-23T13:17:44.555951+0000","last_became_active":"2024-03-23T13:17:40.235810+0000","last_became_peered":"2024-03-23T13:17:40.235810+0000","last_unstale":"2024-03-23T13:17:44.555951+0000","last_undegraded":"2024-03-23T13:17:44.555951+0000","last_fullsized":"2024-03-23T13:17:44.555951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:27:28.193164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809150+0000","last_change":"2024-03-23T13:17:41.252261+0000","last_active":"2024-03-23T13:17:42.809150+0000","last_peered":"2024-03-23T13:17:42.809150+0000","last_clean":"2024-03-23T13:17:42.809150+0000","last_became_active":"2024-03-23T13:17:41.251949+0000","last_became_peered":"2024-03-23T13:17:41.251949+0000","last_unstale":"2024-03-23T13:17:42.809150+0000","last_undegraded":"2024-03-23T13:17:42.809150+0000","last_fullsized":"2024-03-23T13:17:42.809150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:39.576999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.560996+0000","last_change":"2024-03-23T13:17:40.228738+0000","last_active":"2024-03-23T13:17:45.560996+0000","last_peered":"2024-03-23T13:17:45.560996+0000","last_clean":"2024-03-23T13:17:45.560996+0000","last_became_active":"2024-03-23T13:17:40.228424+0000","last_became_peered":"2024-03-23T13:17:40.228424+0000","last_unstale":"2024-03-23T13:17:45.560996+0000","last_undegraded":"2024-03-23T13:17:45.560996+0000","last_fullsized":"2024-03-23T13:17:45.560996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:51.506223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.808815+0000","last_change":"2024-03-23T13:17:41.247155+0000","last_active":"2024-03-23T13:17:42.808815+0000","last_peered":"2024-03-23T13:17:42.808815+0000","last_clean":"2024-03-23T13:17:42.808815+0000","last_became_active":"2024-03-23T13:17:41.246849+0000","last_became_peered":"2024-03-23T13:17:41.246849+0000","last_unstale":"2024-03-23T13:17:42.808815+0000","last_undegraded":"2024-03-23T13:17:42.808815+0000","last_fullsized":"2024-03-23T13:17:42.808815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:23.311456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.564467+0000","last_change":"2024-03-23T13:17:40.232420+0000","last_active":"2024-03-23T13:17:45.564467+0000","last_peered":"2024-03-23T13:17:45.564467+0000","last_clean":"2024-03-23T13:17:45.564467+0000","last_became_active":"2024-03-23T13:17:40.231940+0000","last_became_peered":"2024-03-23T13:17:40.231940+0000","last_unstale":"2024-03-23T13:17:45.564467+0000","last_undegraded":"2024-03-23T13:17:45.564467+0000","last_fullsized":"2024-03-23T13:17:45.564467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:46.300579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621609+0000","last_change":"2024-03-23T13:17:41.249717+0000","last_active":"2024-03-23T13:17:42.621609+0000","last_peered":"2024-03-23T13:17:42.621609+0000","last_clean":"2024-03-23T13:17:42.621609+0000","last_became_active":"2024-03-23T13:17:41.249286+0000","last_became_peered":"2024-03-23T13:17:41.249286+0000","last_unstale":"2024-03-23T13:17:42.621609+0000","last_undegraded":"2024-03-23T13:17:42.621609+0000","last_fullsized":"2024-03-23T13:17:42.621609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:47:19.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622212+0000","last_change":"2024-03-23T13:17:40.229172+0000","last_active":"2024-03-23T13:17:42.622212+0000","last_peered":"2024-03-23T13:17:42.622212+0000","last_clean":"2024-03-23T13:17:42.622212+0000","last_became_active":"2024-03-23T13:17:40.228858+0000","last_became_peered":"2024-03-23T13:17:40.228858+0000","last_unstale":"2024-03-23T13:17:42.622212+0000","last_undegraded":"2024-03-23T13:17:42.622212+0000","last_fullsized":"2024-03-23T13:17:42.622212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:17.929936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622306+0000","last_change":"2024-03-23T13:17:41.253041+0000","last_active":"2024-03-23T13:17:42.622306+0000","last_peered":"2024-03-23T13:17:42.622306+0000","last_clean":"2024-03-23T13:17:42.622306+0000","last_became_active":"2024-03-23T13:17:41.252739+0000","last_became_peered":"2024-03-23T13:17:41.252739+0000","last_unstale":"2024-03-23T13:17:42.622306+0000","last_undegraded":"2024-03-23T13:17:42.622306+0000","last_fullsized":"2024-03-23T13:17:42.622306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:27:52.560688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.692522+0000","last_change":"2024-03-23T13:17:40.228098+0000","last_active":"2024-03-23T13:17:42.692522+0000","last_peered":"2024-03-23T13:17:42.692522+0000","last_clean":"2024-03-23T13:17:42.692522+0000","last_became_active":"2024-03-23T13:17:40.227778+0000","last_became_peered":"2024-03-23T13:17:40.227778+0000","last_unstale":"2024-03-23T13:17:42.692522+0000","last_undegraded":"2024-03-23T13:17:42.692522+0000","last_fullsized":"2024-03-23T13:17:42.692522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:25:38.862329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.540610+0000","last_change":"2024-03-23T13:17:41.242666+0000","last_active":"2024-03-23T13:17:42.540610+0000","last_peered":"2024-03-23T13:17:42.540610+0000","last_clean":"2024-03-23T13:17:42.540610+0000","last_became_active":"2024-03-23T13:17:41.242371+0000","last_became_peered":"2024-03-23T13:17:41.242371+0000","last_unstale":"2024-03-23T13:17:42.540610+0000","last_undegraded":"2024-03-23T13:17:42.540610+0000","last_fullsized":"2024-03-23T13:17:42.540610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:40.870090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.808896+0000","last_change":"2024-03-23T13:17:40.233558+0000","last_active":"2024-03-23T13:17:42.808896+0000","last_peered":"2024-03-23T13:17:42.808896+0000","last_clean":"2024-03-23T13:17:42.808896+0000","last_became_active":"2024-03-23T13:17:40.232112+0000","last_became_peered":"2024-03-23T13:17:40.232112+0000","last_unstale":"2024-03-23T13:17:42.808896+0000","last_undegraded":"2024-03-23T13:17:42.808896+0000","last_fullsized":"2024-03-23T13:17:42.808896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:37:43.971598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.540634+0000","last_change":"2024-03-23T13:17:41.245804+0000","last_active":"2024-03-23T13:17:42.540634+0000","last_peered":"2024-03-23T13:17:42.540634+0000","last_clean":"2024-03-23T13:17:42.540634+0000","last_became_active":"2024-03-23T13:17:41.245509+0000","last_became_peered":"2024-03-23T13:17:41.245509+0000","last_unstale":"2024-03-23T13:17:42.540634+0000","last_undegraded":"2024-03-23T13:17:42.540634+0000","last_fullsized":"2024-03-23T13:17:42.540634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:39:55.735413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621823+0000","last_change":"2024-03-23T13:17:41.250553+0000","last_active":"2024-03-23T13:17:42.621823+0000","last_peered":"2024-03-23T13:17:42.621823+0000","last_clean":"2024-03-23T13:17:42.621823+0000","last_became_active":"2024-03-23T13:17:41.250140+0000","last_became_peered":"2024-03-23T13:17:41.250140+0000","last_unstale":"2024-03-23T13:17:42.621823+0000","last_undegraded":"2024-03-23T13:17:42.621823+0000","last_fullsized":"2024-03-23T13:17:42.621823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:28.088801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.563450+0000","last_change":"2024-03-23T13:17:40.239651+0000","last_active":"2024-03-23T13:17:45.563450+0000","last_peered":"2024-03-23T13:17:45.563450+0000","last_clean":"2024-03-23T13:17:45.563450+0000","last_became_active":"2024-03-23T13:17:40.239326+0000","last_became_peered":"2024-03-23T13:17:40.239326+0000","last_unstale":"2024-03-23T13:17:45.563450+0000","last_undegraded":"2024-03-23T13:17:45.563450+0000","last_fullsized":"2024-03-23T13:17:45.563450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:56:36.147882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621480+0000","last_change":"2024-03-23T13:17:41.253148+0000","last_active":"2024-03-23T13:17:42.621480+0000","last_peered":"2024-03-23T13:17:42.621480+0000","last_clean":"2024-03-23T13:17:42.621480+0000","last_became_active":"2024-03-23T13:17:41.252571+0000","last_became_peered":"2024-03-23T13:17:41.252571+0000","last_unstale":"2024-03-23T13:17:42.621480+0000","last_undegraded":"2024-03-23T13:17:42.621480+0000","last_fullsized":"2024-03-23T13:17:42.621480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:19:36.477844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.555550+0000","last_change":"2024-03-23T13:17:40.227457+0000","last_active":"2024-03-23T13:17:44.555550+0000","last_peered":"2024-03-23T13:17:44.555550+0000","last_clean":"2024-03-23T13:17:44.555550+0000","last_became_active":"2024-03-23T13:17:40.227149+0000","last_became_peered":"2024-03-23T13:17:40.227149+0000","last_unstale":"2024-03-23T13:17:44.555550+0000","last_undegraded":"2024-03-23T13:17:44.555550+0000","last_fullsized":"2024-03-23T13:17:44.555550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:33:20.606023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":337,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.808730+0000","last_change":"2024-03-23T13:16:31.257961+0000","last_active":"2024-03-23T13:17:42.808730+0000","last_peered":"2024-03-23T13:17:42.808730+0000","last_clean":"2024-03-23T13:17:42.808730+0000","last_became_active":"2024-03-23T13:16:31.257444+0000","last_became_peered":"2024-03-23T13:16:31.257444+0000","last_unstale":"2024-03-23T13:17:42.808730+0000","last_undegraded":"2024-03-23T13:17:42.808730+0000","last_fullsized":"2024-03-23T13:17:42.808730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:13:08.847194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:13:08.847194+0000","last_clean_scrub_stamp":"2024-03-23T13:13:08.847194+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:35:07.006029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234126+0000","last_change":"2024-03-23T13:17:41.249443+0000","last_active":"2024-03-23T13:17:42.234126+0000","last_peered":"2024-03-23T13:17:42.234126+0000","last_clean":"2024-03-23T13:17:42.234126+0000","last_became_active":"2024-03-23T13:17:41.248960+0000","last_became_peered":"2024-03-23T13:17:41.248960+0000","last_unstale":"2024-03-23T13:17:42.234126+0000","last_undegraded":"2024-03-23T13:17:42.234126+0000","last_fullsized":"2024-03-23T13:17:42.234126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:35:36.819271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.808951+0000","last_change":"2024-03-23T13:17:40.230268+0000","last_active":"2024-03-23T13:17:42.808951+0000","last_peered":"2024-03-23T13:17:42.808951+0000","last_clean":"2024-03-23T13:17:42.808951+0000","last_became_active":"2024-03-23T13:17:40.229955+0000","last_became_peered":"2024-03-23T13:17:40.229955+0000","last_unstale":"2024-03-23T13:17:42.808951+0000","last_undegraded":"2024-03-23T13:17:42.808951+0000","last_fullsized":"2024-03-23T13:17:42.808951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:38.977844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:46.574934+0000","last_change":"2024-03-23T13:17:40.235259+0000","last_active":"2024-03-23T13:17:46.574934+0000","last_peered":"2024-03-23T13:17:46.574934+0000","last_clean":"2024-03-23T13:17:46.574934+0000","last_became_active":"2024-03-23T13:17:40.234941+0000","last_became_peered":"2024-03-23T13:17:40.234941+0000","last_unstale":"2024-03-23T13:17:46.574934+0000","last_undegraded":"2024-03-23T13:17:46.574934+0000","last_fullsized":"2024-03-23T13:17:46.574934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:31:24.799836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234441+0000","last_change":"2024-03-23T13:17:41.248807+0000","last_active":"2024-03-23T13:17:42.234441+0000","last_peered":"2024-03-23T13:17:42.234441+0000","last_clean":"2024-03-23T13:17:42.234441+0000","last_became_active":"2024-03-23T13:17:41.248401+0000","last_became_peered":"2024-03-23T13:17:41.248401+0000","last_unstale":"2024-03-23T13:17:42.234441+0000","last_undegraded":"2024-03-23T13:17:42.234441+0000","last_fullsized":"2024-03-23T13:17:42.234441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:13:30.533996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:46.575596+0000","last_change":"2024-03-23T13:17:40.233323+0000","last_active":"2024-03-23T13:17:46.575596+0000","last_peered":"2024-03-23T13:17:46.575596+0000","last_clean":"2024-03-23T13:17:46.575596+0000","last_became_active":"2024-03-23T13:17:40.233031+0000","last_became_peered":"2024-03-23T13:17:40.233031+0000","last_unstale":"2024-03-23T13:17:46.575596+0000","last_undegraded":"2024-03-23T13:17:46.575596+0000","last_fullsized":"2024-03-23T13:17:46.575596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:55:35.870633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621803+0000","last_change":"2024-03-23T13:17:41.243662+0000","last_active":"2024-03-23T13:17:42.621803+0000","last_peered":"2024-03-23T13:17:42.621803+0000","last_clean":"2024-03-23T13:17:42.621803+0000","last_became_active":"2024-03-23T13:17:41.243310+0000","last_became_peered":"2024-03-23T13:17:41.243310+0000","last_unstale":"2024-03-23T13:17:42.621803+0000","last_undegraded":"2024-03-23T13:17:42.621803+0000","last_fullsized":"2024-03-23T13:17:42.621803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:25:06.277185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.553499+0000","last_change":"2024-03-23T13:17:40.226698+0000","last_active":"2024-03-23T13:17:44.553499+0000","last_peered":"2024-03-23T13:17:44.553499+0000","last_clean":"2024-03-23T13:17:44.553499+0000","last_became_active":"2024-03-23T13:17:40.226305+0000","last_became_peered":"2024-03-23T13:17:40.226305+0000","last_unstale":"2024-03-23T13:17:44.553499+0000","last_undegraded":"2024-03-23T13:17:44.553499+0000","last_fullsized":"2024-03-23T13:17:44.553499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:06:19.529125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585342+0000","last_change":"2024-03-23T13:17:41.247685+0000","last_active":"2024-03-23T13:17:42.585342+0000","last_peered":"2024-03-23T13:17:42.585342+0000","last_clean":"2024-03-23T13:17:42.585342+0000","last_became_active":"2024-03-23T13:17:41.247389+0000","last_became_peered":"2024-03-23T13:17:41.247389+0000","last_unstale":"2024-03-23T13:17:42.585342+0000","last_undegraded":"2024-03-23T13:17:42.585342+0000","last_fullsized":"2024-03-23T13:17:42.585342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:53.258831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.555358+0000","last_change":"2024-03-23T13:17:40.228860+0000","last_active":"2024-03-23T13:17:44.555358+0000","last_peered":"2024-03-23T13:17:44.555358+0000","last_clean":"2024-03-23T13:17:44.555358+0000","last_became_active":"2024-03-23T13:17:40.228535+0000","last_became_peered":"2024-03-23T13:17:40.228535+0000","last_unstale":"2024-03-23T13:17:44.555358+0000","last_undegraded":"2024-03-23T13:17:44.555358+0000","last_fullsized":"2024-03-23T13:17:44.555358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:05:33.739369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621759+0000","last_change":"2024-03-23T13:17:41.239461+0000","last_active":"2024-03-23T13:17:42.621759+0000","last_peered":"2024-03-23T13:17:42.621759+0000","last_clean":"2024-03-23T13:17:42.621759+0000","last_became_active":"2024-03-23T13:17:41.239137+0000","last_became_peered":"2024-03-23T13:17:41.239137+0000","last_unstale":"2024-03-23T13:17:42.621759+0000","last_undegraded":"2024-03-23T13:17:42.621759+0000","last_fullsized":"2024-03-23T13:17:42.621759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:06:43.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.554711+0000","last_change":"2024-03-23T13:17:40.237796+0000","last_active":"2024-03-23T13:17:44.554711+0000","last_peered":"2024-03-23T13:17:44.554711+0000","last_clean":"2024-03-23T13:17:44.554711+0000","last_became_active":"2024-03-23T13:17:40.236596+0000","last_became_peered":"2024-03-23T13:17:40.236596+0000","last_unstale":"2024-03-23T13:17:44.554711+0000","last_undegraded":"2024-03-23T13:17:44.554711+0000","last_fullsized":"2024-03-23T13:17:44.554711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:22.767459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621439+0000","last_change":"2024-03-23T13:17:41.250379+0000","last_active":"2024-03-23T13:17:42.621439+0000","last_peered":"2024-03-23T13:17:42.621439+0000","last_clean":"2024-03-23T13:17:42.621439+0000","last_became_active":"2024-03-23T13:17:41.244850+0000","last_became_peered":"2024-03-23T13:17:41.244850+0000","last_unstale":"2024-03-23T13:17:42.621439+0000","last_undegraded":"2024-03-23T13:17:42.621439+0000","last_fullsized":"2024-03-23T13:17:42.621439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:34:42.741865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:45.551310+0000","last_change":"2024-03-23T13:17:40.234334+0000","last_active":"2024-03-23T13:17:45.551310+0000","last_peered":"2024-03-23T13:17:45.551310+0000","last_clean":"2024-03-23T13:17:45.551310+0000","last_became_active":"2024-03-23T13:17:40.233953+0000","last_became_peered":"2024-03-23T13:17:40.233953+0000","last_unstale":"2024-03-23T13:17:45.551310+0000","last_undegraded":"2024-03-23T13:17:45.551310+0000","last_fullsized":"2024-03-23T13:17:45.551310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:53:48.238091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621770+0000","last_change":"2024-03-23T13:17:41.247960+0000","last_active":"2024-03-23T13:17:42.621770+0000","last_peered":"2024-03-23T13:17:42.621770+0000","last_clean":"2024-03-23T13:17:42.621770+0000","last_became_active":"2024-03-23T13:17:41.247459+0000","last_became_peered":"2024-03-23T13:17:41.247459+0000","last_unstale":"2024-03-23T13:17:42.621770+0000","last_undegraded":"2024-03-23T13:17:42.621770+0000","last_fullsized":"2024-03-23T13:17:42.621770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:01.345486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:44.556504+0000","last_change":"2024-03-23T13:17:40.237228+0000","last_active":"2024-03-23T13:17:44.556504+0000","last_peered":"2024-03-23T13:17:44.556504+0000","last_clean":"2024-03-23T13:17:44.556504+0000","last_became_active":"2024-03-23T13:17:40.236843+0000","last_became_peered":"2024-03-23T13:17:40.236843+0000","last_unstale":"2024-03-23T13:17:44.556504+0000","last_undegraded":"2024-03-23T13:17:44.556504+0000","last_fullsized":"2024-03-23T13:17:44.556504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:31:53.315272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809091+0000","last_change":"2024-03-23T13:17:41.256459+0000","last_active":"2024-03-23T13:17:42.809091+0000","last_peered":"2024-03-23T13:17:42.809091+0000","last_clean":"2024-03-23T13:17:42.809091+0000","last_became_active":"2024-03-23T13:17:41.256168+0000","last_became_peered":"2024-03-23T13:17:41.256168+0000","last_unstale":"2024-03-23T13:17:42.809091+0000","last_undegraded":"2024-03-23T13:17:42.809091+0000","last_fullsized":"2024-03-23T13:17:42.809091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:50.488371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:46.573487+0000","last_change":"2024-03-23T13:17:40.224068+0000","last_active":"2024-03-23T13:17:46.573487+0000","last_peered":"2024-03-23T13:17:46.573487+0000","last_clean":"2024-03-23T13:17:46.573487+0000","last_became_active":"2024-03-23T13:17:40.223744+0000","last_became_peered":"2024-03-23T13:17:40.223744+0000","last_unstale":"2024-03-23T13:17:46.573487+0000","last_undegraded":"2024-03-23T13:17:46.573487+0000","last_fullsized":"2024-03-23T13:17:46.573487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:08:56.187695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809207+0000","last_change":"2024-03-23T13:17:41.257665+0000","last_active":"2024-03-23T13:17:42.809207+0000","last_peered":"2024-03-23T13:17:42.809207+0000","last_clean":"2024-03-23T13:17:42.809207+0000","last_became_active":"2024-03-23T13:17:41.256943+0000","last_became_peered":"2024-03-23T13:17:41.256943+0000","last_unstale":"2024-03-23T13:17:42.809207+0000","last_undegraded":"2024-03-23T13:17:42.809207+0000","last_fullsized":"2024-03-23T13:17:42.809207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:12:21.896800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622031+0000","last_change":"2024-03-23T13:17:40.236231+0000","last_active":"2024-03-23T13:17:42.622031+0000","last_peered":"2024-03-23T13:17:42.622031+0000","last_clean":"2024-03-23T13:17:42.622031+0000","last_became_active":"2024-03-23T13:17:40.235880+0000","last_became_peered":"2024-03-23T13:17:40.235880+0000","last_unstale":"2024-03-23T13:17:42.622031+0000","last_undegraded":"2024-03-23T13:17:42.622031+0000","last_fullsized":"2024-03-23T13:17:42.622031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:38.400890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809066+0000","last_change":"2024-03-23T13:17:41.252326+0000","last_active":"2024-03-23T13:17:42.809066+0000","last_peered":"2024-03-23T13:17:42.809066+0000","last_clean":"2024-03-23T13:17:42.809066+0000","last_became_active":"2024-03-23T13:17:41.252023+0000","last_became_peered":"2024-03-23T13:17:41.252023+0000","last_unstale":"2024-03-23T13:17:42.809066+0000","last_undegraded":"2024-03-23T13:17:42.809066+0000","last_fullsized":"2024-03-23T13:17:42.809066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:18:51.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234383+0000","last_change":"2024-03-23T13:17:40.239647+0000","last_active":"2024-03-23T13:17:42.234383+0000","last_peered":"2024-03-23T13:17:42.234383+0000","last_clean":"2024-03-23T13:17:42.234383+0000","last_became_active":"2024-03-23T13:17:40.239318+0000","last_became_peered":"2024-03-23T13:17:40.239318+0000","last_unstale":"2024-03-23T13:17:42.234383+0000","last_undegraded":"2024-03-23T13:17:42.234383+0000","last_fullsized":"2024-03-23T13:17:42.234383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:56:03.354003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.809154+0000","last_change":"2024-03-23T13:17:41.255446+0000","last_active":"2024-03-23T13:17:42.809154+0000","last_peered":"2024-03-23T13:17:42.809154+0000","last_clean":"2024-03-23T13:17:42.809154+0000","last_became_active":"2024-03-23T13:17:41.255135+0000","last_became_peered":"2024-03-23T13:17:41.255135+0000","last_unstale":"2024-03-23T13:17:42.809154+0000","last_undegraded":"2024-03-23T13:17:42.809154+0000","last_fullsized":"2024-03-23T13:17:42.809154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:47.639958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621468+0000","last_change":"2024-03-23T13:17:41.245392+0000","last_active":"2024-03-23T13:17:42.621468+0000","last_peered":"2024-03-23T13:17:42.621468+0000","last_clean":"2024-03-23T13:17:42.621468+0000","last_became_active":"2024-03-23T13:17:41.244439+0000","last_became_peered":"2024-03-23T13:17:41.244439+0000","last_unstale":"2024-03-23T13:17:42.621468+0000","last_undegraded":"2024-03-23T13:17:42.621468+0000","last_fullsized":"2024-03-23T13:17:42.621468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:01:50.028867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234631+0000","last_change":"2024-03-23T13:17:41.253214+0000","last_active":"2024-03-23T13:17:42.234631+0000","last_peered":"2024-03-23T13:17:42.234631+0000","last_clean":"2024-03-23T13:17:42.234631+0000","last_became_active":"2024-03-23T13:17:41.252892+0000","last_became_peered":"2024-03-23T13:17:41.252892+0000","last_unstale":"2024-03-23T13:17:42.234631+0000","last_undegraded":"2024-03-23T13:17:42.234631+0000","last_fullsized":"2024-03-23T13:17:42.234631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:24:15.475541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234582+0000","last_change":"2024-03-23T13:17:41.247561+0000","last_active":"2024-03-23T13:17:42.234582+0000","last_peered":"2024-03-23T13:17:42.234582+0000","last_clean":"2024-03-23T13:17:42.234582+0000","last_became_active":"2024-03-23T13:17:41.247238+0000","last_became_peered":"2024-03-23T13:17:41.247238+0000","last_unstale":"2024-03-23T13:17:42.234582+0000","last_undegraded":"2024-03-23T13:17:42.234582+0000","last_fullsized":"2024-03-23T13:17:42.234582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:37:19.301021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.585201+0000","last_change":"2024-03-23T13:17:41.241607+0000","last_active":"2024-03-23T13:17:42.585201+0000","last_peered":"2024-03-23T13:17:42.585201+0000","last_clean":"2024-03-23T13:17:42.585201+0000","last_became_active":"2024-03-23T13:17:41.241285+0000","last_became_peered":"2024-03-23T13:17:41.241285+0000","last_unstale":"2024-03-23T13:17:42.585201+0000","last_undegraded":"2024-03-23T13:17:42.585201+0000","last_fullsized":"2024-03-23T13:17:42.585201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:34:35.800087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234420+0000","last_change":"2024-03-23T13:17:41.245423+0000","last_active":"2024-03-23T13:17:42.234420+0000","last_peered":"2024-03-23T13:17:42.234420+0000","last_clean":"2024-03-23T13:17:42.234420+0000","last_became_active":"2024-03-23T13:17:41.245051+0000","last_became_peered":"2024-03-23T13:17:41.245051+0000","last_unstale":"2024-03-23T13:17:42.234420+0000","last_undegraded":"2024-03-23T13:17:42.234420+0000","last_fullsized":"2024-03-23T13:17:42.234420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:10:23.191491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.622049+0000","last_change":"2024-03-23T13:17:41.241371+0000","last_active":"2024-03-23T13:17:42.622049+0000","last_peered":"2024-03-23T13:17:42.622049+0000","last_clean":"2024-03-23T13:17:42.622049+0000","last_became_active":"2024-03-23T13:17:41.240970+0000","last_became_peered":"2024-03-23T13:17:41.240970+0000","last_unstale":"2024-03-23T13:17:42.622049+0000","last_undegraded":"2024-03-23T13:17:42.622049+0000","last_fullsized":"2024-03-23T13:17:42.622049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:14.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.235098+0000","last_change":"2024-03-23T13:17:41.248578+0000","last_active":"2024-03-23T13:17:42.235098+0000","last_peered":"2024-03-23T13:17:42.235098+0000","last_clean":"2024-03-23T13:17:42.235098+0000","last_became_active":"2024-03-23T13:17:41.248264+0000","last_became_peered":"2024-03-23T13:17:41.248264+0000","last_unstale":"2024-03-23T13:17:42.235098+0000","last_undegraded":"2024-03-23T13:17:42.235098+0000","last_fullsized":"2024-03-23T13:17:42.235098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:42.937816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621682+0000","last_change":"2024-03-23T13:17:41.246138+0000","last_active":"2024-03-23T13:17:42.621682+0000","last_peered":"2024-03-23T13:17:42.621682+0000","last_clean":"2024-03-23T13:17:42.621682+0000","last_became_active":"2024-03-23T13:17:41.245596+0000","last_became_peered":"2024-03-23T13:17:41.245596+0000","last_unstale":"2024-03-23T13:17:42.621682+0000","last_undegraded":"2024-03-23T13:17:42.621682+0000","last_fullsized":"2024-03-23T13:17:42.621682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:51:43.524257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621437+0000","last_change":"2024-03-23T13:17:41.246819+0000","last_active":"2024-03-23T13:17:42.621437+0000","last_peered":"2024-03-23T13:17:42.621437+0000","last_clean":"2024-03-23T13:17:42.621437+0000","last_became_active":"2024-03-23T13:17:41.246389+0000","last_became_peered":"2024-03-23T13:17:41.246389+0000","last_unstale":"2024-03-23T13:17:42.621437+0000","last_undegraded":"2024-03-23T13:17:42.621437+0000","last_fullsized":"2024-03-23T13:17:42.621437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:40.759387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621230+0000","last_change":"2024-03-23T13:17:41.254185+0000","last_active":"2024-03-23T13:17:42.621230+0000","last_peered":"2024-03-23T13:17:42.621230+0000","last_clean":"2024-03-23T13:17:42.621230+0000","last_became_active":"2024-03-23T13:17:41.253828+0000","last_became_peered":"2024-03-23T13:17:41.253828+0000","last_unstale":"2024-03-23T13:17:42.621230+0000","last_undegraded":"2024-03-23T13:17:42.621230+0000","last_fullsized":"2024-03-23T13:17:42.621230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:27.826109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.236101+0000","last_change":"2024-03-23T13:17:41.257669+0000","last_active":"2024-03-23T13:17:42.236101+0000","last_peered":"2024-03-23T13:17:42.236101+0000","last_clean":"2024-03-23T13:17:42.236101+0000","last_became_active":"2024-03-23T13:17:41.257341+0000","last_became_peered":"2024-03-23T13:17:41.257341+0000","last_unstale":"2024-03-23T13:17:42.236101+0000","last_undegraded":"2024-03-23T13:17:42.236101+0000","last_fullsized":"2024-03-23T13:17:42.236101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:58:08.777906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.234551+0000","last_change":"2024-03-23T13:17:41.249078+0000","last_active":"2024-03-23T13:17:42.234551+0000","last_peered":"2024-03-23T13:17:42.234551+0000","last_clean":"2024-03-23T13:17:42.234551+0000","last_became_active":"2024-03-23T13:17:41.247804+0000","last_became_peered":"2024-03-23T13:17:41.247804+0000","last_unstale":"2024-03-23T13:17:42.234551+0000","last_undegraded":"2024-03-23T13:17:42.234551+0000","last_fullsized":"2024-03-23T13:17:42.234551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:20:19.514109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T13:17:42.621764+0000","last_change":"2024-03-23T13:17:41.242707+0000","last_active":"2024-03-23T13:17:42.621764+0000","last_peered":"2024-03-23T13:17:42.621764+0000","last_clean":"2024-03-23T13:17:42.621764+0000","last_became_active":"2024-03-23T13:17:41.242367+0000","last_became_peered":"2024-03-23T13:17:41.242367+0000","last_unstale":"2024-03-23T13:17:42.621764+0000","last_undegraded":"2024-03-23T13:17:42.621764+0000","last_fullsized":"2024-03-23T13:17:42.621764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:14:30.695624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249559,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6415,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":20307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1032633,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":439257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":438873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332129,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":1029085,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":438225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":440329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":440313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":441097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":438329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920816,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":438779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":1029979,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34482,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34482,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T13:17:51.050 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T13:17:51.050 INFO:tasks.cephadm:Setting up client nodes... 2024-03-23T13:17:51.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-23T13:17:51.327 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:17:51.730 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:51 smithi007 ceph-mon[32184]: pgmap v289: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s wr, 17 op/s 2024-03-23T13:17:51.730 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:51 smithi007 ceph-mon[32184]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:17:51.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:51 smithi130 ceph-mon[36225]: pgmap v289: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s wr, 17 op/s 2024-03-23T13:17:51.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:51 smithi130 ceph-mon[36225]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:17:52.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:51 smithi155 ceph-mon[36157]: pgmap v289: 97 pgs: 97 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s wr, 17 op/s 2024-03-23T13:17:52.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:51 smithi155 ceph-mon[36157]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:17:52.582 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:52 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: 2024-03-23T13:17:52.201+0000 7fead4ed0700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-23T13:17:52.582 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:17:52 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: 2024-03-23T13:17:52.201+0000 7fead4ed0700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-23T13:17:52.815 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:52 smithi007 ceph-mon[32184]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:17:52.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:52 smithi130 ceph-mon[36225]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:17:53.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:52 smithi155 ceph-mon[36157]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:17:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.197+0000 7f491dea8700 1 -- 172.21.15.7:0/1778742030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4918103470 msgr2=0x7f49181038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.197+0000 7f491dea8700 1 --2- 172.21.15.7:0/1778742030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4918103470 0x7f49181038d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f490c0099c0 tx=0x7f490c009cd0 comp rx=0 tx=0).stop 2024-03-23T13:17:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.197+0000 7f491dea8700 1 -- 172.21.15.7:0/1778742030 shutdown_connections 2024-03-23T13:17:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.197+0000 7f491dea8700 1 --2- 172.21.15.7:0/1778742030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4918103e10 0x7f491810a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.197+0000 7f491dea8700 1 --2- 172.21.15.7:0/1778742030 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4918102270 0x7f4918102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.197+0000 7f491dea8700 1 --2- 172.21.15.7:0/1778742030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4918103470 0x7f49181038d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.197+0000 7f491dea8700 1 -- 172.21.15.7:0/1778742030 >> 172.21.15.7:0/1778742030 conn(0x7f49180fd850 msgr2=0x7f49180ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.197+0000 7f491dea8700 1 -- 172.21.15.7:0/1778742030 shutdown_connections 2024-03-23T13:17:53.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.197+0000 7f491dea8700 1 -- 172.21.15.7:0/1778742030 wait complete. 2024-03-23T13:17:53.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f491dea8700 1 Processor -- start 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f491dea8700 1 -- start start 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f491dea8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4918102270 0x7f4918198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f491dea8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4918103470 0x7f4918198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f491dea8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4918103e10 0x7f491819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f491dea8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f491819fcb0 con 0x7f4918103e10 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f491dea8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f491819fdf0 con 0x7f4918102270 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f491dea8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f491819ff80 con 0x7f4918103470 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f4917fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4918103e10 0x7f491819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f49177fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4918102270 0x7f4918198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f4916ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4918103470 0x7f4918198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f4917fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4918103e10 0x7f491819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52704/0 (socket says 172.21.15.7:52704) 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f49177fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4918102270 0x7f4918198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:38102/0 (socket says 172.21.15.7:38102) 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.198+0000 7f4917fff700 1 -- 172.21.15.7:0/2413651875 learned_addr learned my addr 172.21.15.7:0/2413651875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f4917fff700 1 -- 172.21.15.7:0/2413651875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4918103470 msgr2=0x7f4918198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f4917fff700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4918103470 0x7f4918198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f4917fff700 1 -- 172.21.15.7:0/2413651875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4918102270 msgr2=0x7f4918198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f4917fff700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4918102270 0x7f4918198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f4917fff700 1 -- 172.21.15.7:0/2413651875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c0095f0 con 0x7f4918103e10 2024-03-23T13:17:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f4917fff700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4918103e10 0x7f491819f6e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f491000b400 tx=0x7f491000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f4914ff9700 1 -- 172.21.15.7:0/2413651875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910015070 con 0x7f4918103e10 2024-03-23T13:17:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f4914ff9700 1 -- 172.21.15.7:0/2413651875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f491000be40 con 0x7f4918103e10 2024-03-23T13:17:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f491dea8700 1 -- 172.21.15.7:0/2413651875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4918074490 con 0x7f4918103e10 2024-03-23T13:17:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f4914ff9700 1 -- 172.21.15.7:0/2413651875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910007da0 con 0x7f4918103e10 2024-03-23T13:17:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.199+0000 7f491dea8700 1 -- 172.21.15.7:0/2413651875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4918074940 con 0x7f4918103e10 2024-03-23T13:17:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.200+0000 7f491dea8700 1 -- 172.21.15.7:0/2413651875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4918109560 con 0x7f4918103e10 2024-03-23T13:17:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.201+0000 7f4914ff9700 1 -- 172.21.15.7:0/2413651875 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f4910004ad0 con 0x7f4918103e10 2024-03-23T13:17:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.202+0000 7f4914ff9700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f49000a2290 0x7f49000a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.202+0000 7f49177fe700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f49000a2290 0x7f49000a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.202+0000 7f4914ff9700 1 -- 172.21.15.7:0/2413651875 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f49100d1580 con 0x7f4918103e10 2024-03-23T13:17:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.202+0000 7f49177fe700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f49000a2290 0x7f49000a4750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f49181032d0 tx=0x7f4908009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:53.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.206+0000 7f4914ff9700 1 -- 172.21.15.7:0/2413651875 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f491009a5a0 con 0x7f4918103e10 2024-03-23T13:17:53.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.492+0000 7f491dea8700 1 -- 172.21.15.7:0/2413651875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f4918050710 con 0x7f4918103e10 2024-03-23T13:17:53.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.494+0000 7f4914ff9700 1 -- 172.21.15.7:0/2413651875 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f491009dc60 con 0x7f4918103e10 2024-03-23T13:17:53.494 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-03-23T13:17:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCB1v5lfxRoHRAAHhM8hHEBskg3/w5XQsmojA== 2024-03-23T13:17:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 -- 172.21.15.7:0/2413651875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f49000a2290 msgr2=0x7f49000a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f49000a2290 0x7f49000a4750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f49181032d0 tx=0x7f4908009260 comp rx=0 tx=0).stop 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 -- 172.21.15.7:0/2413651875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4918103e10 msgr2=0x7f491819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4918103e10 0x7f491819f6e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f491000b400 tx=0x7f491000b710 comp rx=0 tx=0).stop 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 -- 172.21.15.7:0/2413651875 shutdown_connections 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f49000a2290 0x7f49000a4750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4918103e10 0x7f491819f6e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4918102270 0x7f4918198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 --2- 172.21.15.7:0/2413651875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4918103470 0x7f4918198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 -- 172.21.15.7:0/2413651875 >> 172.21.15.7:0/2413651875 conn(0x7f49180fd850 msgr2=0x7f49181066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 -- 172.21.15.7:0/2413651875 shutdown_connections 2024-03-23T13:17:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:53.496+0000 7f491dea8700 1 -- 172.21.15.7:0/2413651875 wait complete. 2024-03-23T13:17:53.809 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:53 smithi007 ceph-mon[32184]: pgmap v290: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5.7 KiB/s wr, 22 op/s 2024-03-23T13:17:53.809 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:53 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2413651875' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T13:17:53.809 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:53 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2413651875' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T13:17:53.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:53 smithi130 ceph-mon[36225]: pgmap v290: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5.7 KiB/s wr, 22 op/s 2024-03-23T13:17:53.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:53 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2413651875' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T13:17:53.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:53 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2413651875' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T13:17:54.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:53 smithi155 ceph-mon[36157]: pgmap v290: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5.7 KiB/s wr, 22 op/s 2024-03-23T13:17:54.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:53 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2413651875' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T13:17:54.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:53 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2413651875' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T13:17:54.131 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:17:54.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-23T13:17:54.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-23T13:17:54.173 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-23T13:17:54.174 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-23T13:17:54.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-23T13:17:54.457 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:17:55.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:55 smithi007 ceph-mon[32184]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.1 KiB/s wr, 25 op/s 2024-03-23T13:17:55.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:55 smithi130 ceph-mon[36225]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.1 KiB/s wr, 25 op/s 2024-03-23T13:17:56.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:55 smithi155 ceph-mon[36157]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.1 KiB/s wr, 25 op/s 2024-03-23T13:17:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.210+0000 7f890f4f7700 1 -- 172.21.15.7:0/674306567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89081024e0 msgr2=0x7f89081028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.210+0000 7f890f4f7700 1 --2- 172.21.15.7:0/674306567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89081024e0 0x7f89081028c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f890400b0a0 tx=0x7f890400b3b0 comp rx=0 tx=0).stop 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.210+0000 7f890f4f7700 1 -- 172.21.15.7:0/674306567 shutdown_connections 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.210+0000 7f890f4f7700 1 --2- 172.21.15.7:0/674306567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908107510 0x7f890810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.210+0000 7f890f4f7700 1 --2- 172.21.15.7:0/674306567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89081024e0 0x7f89081028c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.210+0000 7f890f4f7700 1 --2- 172.21.15.7:0/674306567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8908102e90 0x7f8908106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.210+0000 7f890f4f7700 1 -- 172.21.15.7:0/674306567 >> 172.21.15.7:0/674306567 conn(0x7f8908079290 msgr2=0x7f89080796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.210+0000 7f890f4f7700 1 -- 172.21.15.7:0/674306567 shutdown_connections 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.210+0000 7f890f4f7700 1 -- 172.21.15.7:0/674306567 wait complete. 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.210+0000 7f890f4f7700 1 Processor -- start 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890f4f7700 1 -- start start 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890f4f7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89081024e0 0x7f8908074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890f4f7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908102e90 0x7f8908074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890f4f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908107510 0x7f8908074f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890f4f7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89081a6c60 con 0x7f8908107510 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890f4f7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f89081a6dd0 con 0x7f89081024e0 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890f4f7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f89081a6f90 con 0x7f8908102e90 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890da94700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908107510 0x7f8908074f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890da94700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908107510 0x7f8908074f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56042/0 (socket says 172.21.15.7:56042) 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890ca92700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908102e90 0x7f8908074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890da94700 1 -- 172.21.15.7:0/510644239 learned_addr learned my addr 172.21.15.7:0/510644239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890d293700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89081024e0 0x7f8908074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890da94700 1 -- 172.21.15.7:0/510644239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908102e90 msgr2=0x7f8908074a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890da94700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908102e90 0x7f8908074a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890da94700 1 -- 172.21.15.7:0/510644239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89081024e0 msgr2=0x7f8908074500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890da94700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89081024e0 0x7f8908074500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890da94700 1 -- 172.21.15.7:0/510644239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8904009d00 con 0x7f8908107510 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890ca92700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908102e90 0x7f8908074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:17:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f890da94700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908107510 0x7f8908074f80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f890000e940 tx=0x7f890000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.211+0000 7f88fe7fc700 1 -- 172.21.15.7:0/510644239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89000097b0 con 0x7f8908107510 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.212+0000 7f890f4f7700 1 -- 172.21.15.7:0/510644239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89081a7270 con 0x7f8908107510 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.212+0000 7f88fe7fc700 1 -- 172.21.15.7:0/510644239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8900005020 con 0x7f8908107510 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.212+0000 7f88fe7fc700 1 -- 172.21.15.7:0/510644239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890000ccf0 con 0x7f8908107510 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.212+0000 7f890f4f7700 1 -- 172.21.15.7:0/510644239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89081a7720 con 0x7f8908107510 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.213+0000 7f890f4f7700 1 -- 172.21.15.7:0/510644239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890810a7a0 con 0x7f8908107510 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.214+0000 7f88fe7fc700 1 -- 172.21.15.7:0/510644239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f8900005490 con 0x7f8908107510 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.214+0000 7f88fe7fc700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f88f40a21e0 0x7f88f40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.214+0000 7f890d293700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f88f40a21e0 0x7f88f40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.214+0000 7f88fe7fc700 1 -- 172.21.15.7:0/510644239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f8900014070 con 0x7f8908107510 2024-03-23T13:17:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.215+0000 7f890d293700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f88f40a21e0 0x7f88f40a46a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8904009fd0 tx=0x7f890400b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:56.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.219+0000 7f88fe7fc700 1 -- 172.21.15.7:0/510644239 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f8900096e20 con 0x7f8908107510 2024-03-23T13:17:56.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.481+0000 7f890f4f7700 1 -- 172.21.15.7:0/510644239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8908112250 con 0x7f8908107510 2024-03-23T13:17:56.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.486+0000 7f88fe7fc700 1 -- 172.21.15.7:0/510644239 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+261459 (secure 0 0 0) 0x7f890009a4e0 con 0x7f8908107510 2024-03-23T13:17:56.489 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:56.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 -- 172.21.15.7:0/510644239 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f88f40a21e0 msgr2=0x7f88f40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:56.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f88f40a21e0 0x7f88f40a46a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8904009fd0 tx=0x7f890400b720 comp rx=0 tx=0).stop 2024-03-23T13:17:56.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 -- 172.21.15.7:0/510644239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908107510 msgr2=0x7f8908074f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908107510 0x7f8908074f80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f890000e940 tx=0x7f890000ec50 comp rx=0 tx=0).stop 2024-03-23T13:17:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 -- 172.21.15.7:0/510644239 shutdown_connections 2024-03-23T13:17:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f88f40a21e0 0x7f88f40a46a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908107510 0x7f8908074f80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89081024e0 0x7f8908074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 --2- 172.21.15.7:0/510644239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908102e90 0x7f8908074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 -- 172.21.15.7:0/510644239 >> 172.21.15.7:0/510644239 conn(0x7f8908079290 msgr2=0x7f8908101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 -- 172.21.15.7:0/510644239 shutdown_connections 2024-03-23T13:17:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:56.489+0000 7f890f4f7700 1 -- 172.21.15.7:0/510644239 wait complete. 2024-03-23T13:17:56.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:56 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/510644239' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T13:17:56.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:56 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/510644239' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T13:17:57.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:56 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/510644239' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T13:17:57.234 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":1535208045},{"type":"v1","addr":"172.21.15.7:6801","nonce":1535208045}]},"active_addr":"172.21.15.7:6801/1535208045","active_change":"2024-03-23T13:10:41.843766+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.7:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":797831695}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1567023463}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2162420134}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2108116369}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":232639147}]}]} 2024-03-23T13:17:57.239 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-23T13:17:57.240 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-23T13:17:57.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T13:17:57.490 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:17:57.923 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:57 smithi007 ceph-mon[32184]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-03-23T13:17:57.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:57 smithi130 ceph-mon[36225]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-03-23T13:17:58.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:57 smithi155 ceph-mon[36157]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-03-23T13:17:59.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.170+0000 7f9d1764f700 1 -- 172.21.15.7:0/2167033475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d10107520 msgr2=0x7f9d1010f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.170+0000 7f9d1764f700 1 --2- 172.21.15.7:0/2167033475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d10107520 0x7f9d1010f0f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b0a0 tx=0x7f9d0800b3b0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.170+0000 7f9d1764f700 1 -- 172.21.15.7:0/2167033475 shutdown_connections 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.170+0000 7f9d1764f700 1 --2- 172.21.15.7:0/2167033475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d10107520 0x7f9d1010f0f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.170+0000 7f9d1764f700 1 --2- 172.21.15.7:0/2167033475 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d10102440 0x7f9d10102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.170+0000 7f9d1764f700 1 --2- 172.21.15.7:0/2167033475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d10102df0 0x7f9d10106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.170+0000 7f9d1764f700 1 -- 172.21.15.7:0/2167033475 >> 172.21.15.7:0/2167033475 conn(0x7f9d100fdcb0 msgr2=0x7f9d101000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.170+0000 7f9d1764f700 1 -- 172.21.15.7:0/2167033475 shutdown_connections 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.170+0000 7f9d1764f700 1 -- 172.21.15.7:0/2167033475 wait complete. 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d1764f700 1 Processor -- start 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d1764f700 1 -- start start 2024-03-23T13:17:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d1764f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d10102440 0x7f9d1019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d1764f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d10102df0 0x7f9d1019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d1764f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d10107520 0x7f9d10198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d1764f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10198c80 con 0x7f9d10102df0 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d1764f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10198dc0 con 0x7f9d10107520 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d1764f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10198f80 con 0x7f9d10102440 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d14bea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d10102df0 0x7f9d1019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d153eb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d10102440 0x7f9d1019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d14bea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d10102df0 0x7f9d1019e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56052/0 (socket says 172.21.15.7:56052) 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d153eb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d10102440 0x7f9d1019e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:35886/0 (socket says 172.21.15.7:35886) 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d14bea700 1 -- 172.21.15.7:0/3807786877 learned_addr learned my addr 172.21.15.7:0/3807786877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d15bec700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d10107520 0x7f9d10198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d15bec700 1 -- 172.21.15.7:0/3807786877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d10102440 msgr2=0x7f9d1019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d15bec700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d10102440 0x7f9d1019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d15bec700 1 -- 172.21.15.7:0/3807786877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d10102df0 msgr2=0x7f9d1019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d15bec700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d10102df0 0x7f9d1019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d15bec700 1 -- 172.21.15.7:0/3807786877 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d08009d00 con 0x7f9d10107520 2024-03-23T13:17:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.171+0000 7f9d14bea700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d10102df0 0x7f9d1019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:17:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.172+0000 7f9d15bec700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d10107520 0x7f9d10198620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b0a0 tx=0x7f9d0800bdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.172+0000 7f9d067fc700 1 -- 172.21.15.7:0/3807786877 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800e040 con 0x7f9d10107520 2024-03-23T13:17:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.172+0000 7f9d1764f700 1 -- 172.21.15.7:0/3807786877 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d10199200 con 0x7f9d10107520 2024-03-23T13:17:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.172+0000 7f9d067fc700 1 -- 172.21.15.7:0/3807786877 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d080079d0 con 0x7f9d10107520 2024-03-23T13:17:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.172+0000 7f9d067fc700 1 -- 172.21.15.7:0/3807786877 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08005040 con 0x7f9d10107520 2024-03-23T13:17:59.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.172+0000 7f9d1764f700 1 -- 172.21.15.7:0/3807786877 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d101a5500 con 0x7f9d10107520 2024-03-23T13:17:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.173+0000 7f9d1764f700 1 -- 172.21.15.7:0/3807786877 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1010a840 con 0x7f9d10107520 2024-03-23T13:17:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.174+0000 7f9d067fc700 1 -- 172.21.15.7:0/3807786877 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f9d0801d600 con 0x7f9d10107520 2024-03-23T13:17:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.174+0000 7f9d067fc700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9cfc0a21e0 0x7f9cfc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:17:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.175+0000 7f9d153eb700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9cfc0a21e0 0x7f9cfc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:17:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.175+0000 7f9d067fc700 1 -- 172.21.15.7:0/3807786877 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9d080cf020 con 0x7f9d10107520 2024-03-23T13:17:59.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.175+0000 7f9d153eb700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9cfc0a21e0 0x7f9cfc0a46a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009c60 tx=0x7f9d0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:17:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.184+0000 7f9d067fc700 1 -- 172.21.15.7:0/3807786877 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9d08097fd0 con 0x7f9d10107520 2024-03-23T13:17:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.406+0000 7f9d1764f700 1 -- 172.21.15.7:0/3807786877 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d10199b50 con 0x7f9d10107520 2024-03-23T13:17:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.408+0000 7f9d067fc700 1 -- 172.21.15.7:0/3807786877 <== mon.2 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22124 (secure 0 0 0) 0x7f9d0809b690 con 0x7f9d10107520 2024-03-23T13:17:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:17:59.408 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:17:42.226684+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:17:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 -- 172.21.15.7:0/3807786877 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9cfc0a21e0 msgr2=0x7f9cfc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9cfc0a21e0 0x7f9cfc0a46a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009c60 tx=0x7f9d0c009260 comp rx=0 tx=0).stop 2024-03-23T13:17:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 -- 172.21.15.7:0/3807786877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d10107520 msgr2=0x7f9d10198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:17:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d10107520 0x7f9d10198620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b0a0 tx=0x7f9d0800bdd0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 -- 172.21.15.7:0/3807786877 shutdown_connections 2024-03-23T13:17:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9cfc0a21e0 0x7f9cfc0a46a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d10107520 0x7f9d10198620 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d10102440 0x7f9d1019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 --2- 172.21.15.7:0/3807786877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d10102df0 0x7f9d1019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:17:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 -- 172.21.15.7:0/3807786877 >> 172.21.15.7:0/3807786877 conn(0x7f9d100fdcb0 msgr2=0x7f9d100fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:17:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 -- 172.21.15.7:0/3807786877 shutdown_connections 2024-03-23T13:17:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:17:59.412+0000 7f9d1764f700 1 -- 172.21.15.7:0/3807786877 wait complete. 2024-03-23T13:17:59.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:59 smithi007 ceph-mon[32184]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 25 op/s 2024-03-23T13:17:59.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:17:59 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3807786877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:17:59.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:59 smithi130 ceph-mon[36225]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 25 op/s 2024-03-23T13:17:59.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:17:59 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3807786877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:18:00.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:59 smithi155 ceph-mon[36157]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 25 op/s 2024-03-23T13:18:00.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:17:59 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3807786877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:18:00.096 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-23T13:18:00.096 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T13:18:00.342 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:01.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:01 smithi007 ceph-mon[32184]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-03-23T13:18:01.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:01 smithi130 ceph-mon[36225]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-03-23T13:18:02.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:01 smithi155 ceph-mon[36157]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-03-23T13:18:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.129+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/3399309302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f4102c40 msgr2=0x7fe8f410f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.129+0000 7fe8fb2a1700 1 --2- 172.21.15.7:0/3399309302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f4102c40 0x7fe8f410f0a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00b0a0 tx=0x7fe8ec00b3b0 comp rx=0 tx=0).stop 2024-03-23T13:18:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.129+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/3399309302 shutdown_connections 2024-03-23T13:18:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.129+0000 7fe8fb2a1700 1 --2- 172.21.15.7:0/3399309302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f4102c40 0x7fe8f410f0a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.129+0000 7fe8fb2a1700 1 --2- 172.21.15.7:0/3399309302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8f4108360 0x7fe8f4108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.129+0000 7fe8fb2a1700 1 --2- 172.21.15.7:0/3399309302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f4102280 0x7fe8f4102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.129+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/3399309302 >> 172.21.15.7:0/3399309302 conn(0x7fe8f40792c0 msgr2=0x7fe8f40796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.129+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/3399309302 shutdown_connections 2024-03-23T13:18:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.129+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/3399309302 wait complete. 2024-03-23T13:18:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8fb2a1700 1 Processor -- start 2024-03-23T13:18:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8fb2a1700 1 -- start start 2024-03-23T13:18:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8fb2a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f4102280 0x7fe8f4074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8fb2a1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8f4102c40 0x7fe8f4074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8fb2a1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f4108360 0x7fe8f4074f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8fb2a1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41a6be0 con 0x7fe8f4102280 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8fb2a1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41a6d50 con 0x7fe8f4108360 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8fb2a1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41a6f10 con 0x7fe8f4102c40 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f903d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f4102280 0x7fe8f4074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f903d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f4102280 0x7fe8f4074500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56080/0 (socket says 172.21.15.7:56080) 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f903d700 1 -- 172.21.15.7:0/239374735 learned_addr learned my addr 172.21.15.7:0/239374735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f883c700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8f4102c40 0x7fe8f4074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f983e700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f4108360 0x7fe8f4074f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f903d700 1 -- 172.21.15.7:0/239374735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8f4102c40 msgr2=0x7fe8f4074a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f903d700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8f4102c40 0x7fe8f4074a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f903d700 1 -- 172.21.15.7:0/239374735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f4108360 msgr2=0x7fe8f4074f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f903d700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f4108360 0x7fe8f4074f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f903d700 1 -- 172.21.15.7:0/239374735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec009d00 con 0x7fe8f4102280 2024-03-23T13:18:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f883c700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8f4102c40 0x7fe8f4074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.130+0000 7fe8f983e700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f4108360 0x7fe8f4074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.131+0000 7fe8f903d700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f4102280 0x7fe8f4074500 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000d980 tx=0x7fe8f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.131+0000 7fe8ea7fc700 1 -- 172.21.15.7:0/239374735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0004220 con 0x7fe8f4102280 2024-03-23T13:18:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.131+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/239374735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f41a71f0 con 0x7fe8f4102280 2024-03-23T13:18:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.131+0000 7fe8ea7fc700 1 -- 172.21.15.7:0/239374735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f0007640 con 0x7fe8f4102280 2024-03-23T13:18:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.131+0000 7fe8ea7fc700 1 -- 172.21.15.7:0/239374735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f000f690 con 0x7fe8f4102280 2024-03-23T13:18:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.131+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/239374735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f41a7740 con 0x7fe8f4102280 2024-03-23T13:18:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.132+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/239374735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f410a7f0 con 0x7fe8f4102280 2024-03-23T13:18:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.133+0000 7fe8ea7fc700 1 -- 172.21.15.7:0/239374735 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fe8f0004410 con 0x7fe8f4102280 2024-03-23T13:18:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.133+0000 7fe8ea7fc700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe8e00a21e0 0x7fe8e00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.133+0000 7fe8f883c700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe8e00a21e0 0x7fe8e00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.133+0000 7fe8ea7fc700 1 -- 172.21.15.7:0/239374735 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe8f00cde80 con 0x7fe8f4102280 2024-03-23T13:18:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.134+0000 7fe8f883c700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe8e00a21e0 0x7fe8e00a46a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40797e0 tx=0x7fe8e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.137+0000 7fe8ea7fc700 1 -- 172.21.15.7:0/239374735 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fe8f0096ea0 con 0x7fe8f4102280 2024-03-23T13:18:02.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.341+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/239374735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8f4075a30 con 0x7fe8f4102280 2024-03-23T13:18:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.344+0000 7fe8ea7fc700 1 -- 172.21.15.7:0/239374735 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22124 (secure 0 0 0) 0x7fe8f0016020 con 0x7fe8f4102280 2024-03-23T13:18:02.343 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:18:02.344 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:17:42.226684+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/239374735 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe8e00a21e0 msgr2=0x7fe8e00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe8e00a21e0 0x7fe8e00a46a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40797e0 tx=0x7fe8e4009260 comp rx=0 tx=0).stop 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/239374735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f4102280 msgr2=0x7fe8f4074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f4102280 0x7fe8f4074500 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000d980 tx=0x7fe8f000dc90 comp rx=0 tx=0).stop 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/239374735 shutdown_connections 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe8e00a21e0 0x7fe8e00a46a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f4108360 0x7fe8f4074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f4102280 0x7fe8f4074500 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 --2- 172.21.15.7:0/239374735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8f4102c40 0x7fe8f4074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/239374735 >> 172.21.15.7:0/239374735 conn(0x7fe8f40792c0 msgr2=0x7fe8f4100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/239374735 shutdown_connections 2024-03-23T13:18:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:02.346+0000 7fe8fb2a1700 1 -- 172.21.15.7:0/239374735 wait complete. 2024-03-23T13:18:02.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:02 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/239374735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:18:03.003 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:02 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/239374735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:18:03.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-23T13:18:03.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-23T13:18:03.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-23T13:18:03.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-23T13:18:03.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-23T13:18:03.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-23T13:18:03.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-23T13:18:03.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-23T13:18:03.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-23T13:18:03.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-23T13:18:03.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-23T13:18:03.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-23T13:18:03.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:02 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/239374735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:18:03.672 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.801 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.801 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.808 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.811 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:03.883 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:03 smithi007 ceph-mon[32184]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 19 op/s 2024-03-23T13:18:03.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:03 smithi130 ceph-mon[36225]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 19 op/s 2024-03-23T13:18:04.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:03 smithi155 ceph-mon[36157]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 19 op/s 2024-03-23T13:18:05.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:05 smithi007 ceph-mon[32184]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-03-23T13:18:05.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:05 smithi130 ceph-mon[36225]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-03-23T13:18:06.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:05 smithi155 ceph-mon[36157]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-03-23T13:18:07.758 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:07.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:07 smithi007 ceph-mon[32184]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-23T13:18:07.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:07 smithi130 ceph-mon[36225]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-23T13:18:08.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:07 smithi155 ceph-mon[36157]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-23T13:18:09.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:09 smithi007 ceph-mon[32184]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-23T13:18:09.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:09 smithi130 ceph-mon[36225]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-23T13:18:10.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:09 smithi155 ceph-mon[36157]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-23T13:18:11.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:11 smithi007 ceph-mon[32184]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T13:18:11.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:11 smithi130 ceph-mon[36225]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T13:18:12.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:11 smithi155 ceph-mon[36157]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T13:18:13.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:13 smithi007 ceph-mon[32184]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:18:13.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:13 smithi130 ceph-mon[36225]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:18:14.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:13 smithi155 ceph-mon[36157]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:18:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.314+0000 7f09176bf700 1 -- 172.21.15.7:0/2411133592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910101d70 msgr2=0x7f0910105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.314+0000 7f09176bf700 1 --2- 172.21.15.7:0/2411133592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910101d70 0x7f0910105d30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0900009990 tx=0x7f0900009ca0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.314+0000 7f09176bf700 1 -- 172.21.15.7:0/2411133592 shutdown_connections 2024-03-23T13:18:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.314+0000 7f09176bf700 1 --2- 172.21.15.7:0/2411133592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0910106410 0x7f091010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.314+0000 7f09176bf700 1 --2- 172.21.15.7:0/2411133592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09101013c0 0x7f09101017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.314+0000 7f09176bf700 1 --2- 172.21.15.7:0/2411133592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910101d70 0x7f0910105d30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.314+0000 7f09176bf700 1 -- 172.21.15.7:0/2411133592 >> 172.21.15.7:0/2411133592 conn(0x7f0910077150 msgr2=0x7f0910077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.314+0000 7f09176bf700 1 -- 172.21.15.7:0/2411133592 shutdown_connections 2024-03-23T13:18:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.314+0000 7f09176bf700 1 -- 172.21.15.7:0/2411133592 wait complete. 2024-03-23T13:18:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f09176bf700 1 Processor -- start 2024-03-23T13:18:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f09176bf700 1 -- start start 2024-03-23T13:18:15.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f09176bf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09101013c0 0x7f0910102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f09176bf700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0910101d70 0x7f0910103310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f09176bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910106410 0x7f0910103850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f09176bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09101a6b40 con 0x7f0910106410 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f09176bf700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f09101a6cb0 con 0x7f0910101d70 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f09176bf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f09101a6e70 con 0x7f09101013c0 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f0915c5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910106410 0x7f0910103850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f0915c5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910106410 0x7f0910103850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42214/0 (socket says 172.21.15.7:42214) 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f091545b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09101013c0 0x7f0910102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f0915c5c700 1 -- 172.21.15.7:0/2945513975 learned_addr learned my addr 172.21.15.7:0/2945513975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.315+0000 7f0914c5a700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0910101d70 0x7f0910103310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f0915c5c700 1 -- 172.21.15.7:0/2945513975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09101013c0 msgr2=0x7f0910102dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f0915c5c700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09101013c0 0x7f0910102dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f0915c5c700 1 -- 172.21.15.7:0/2945513975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0910101d70 msgr2=0x7f0910103310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f0915c5c700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0910101d70 0x7f0910103310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f0915c5c700 1 -- 172.21.15.7:0/2945513975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09000095f0 con 0x7f0910106410 2024-03-23T13:18:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f0915c5c700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910106410 0x7f0910103850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f090800ef30 tx=0x7f090800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f09067fc700 1 -- 172.21.15.7:0/2945513975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908018070 con 0x7f0910106410 2024-03-23T13:18:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09101a70f0 con 0x7f0910106410 2024-03-23T13:18:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f09067fc700 1 -- 172.21.15.7:0/2945513975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f090800f040 con 0x7f0910106410 2024-03-23T13:18:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09101a75c0 con 0x7f0910106410 2024-03-23T13:18:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.316+0000 7f09067fc700 1 -- 172.21.15.7:0/2945513975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090800cd50 con 0x7f0910106410 2024-03-23T13:18:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.317+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08f4001030 con 0x7f0910106410 2024-03-23T13:18:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.319+0000 7f09067fc700 1 -- 172.21.15.7:0/2945513975 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f0908004ad0 con 0x7f0910106410 2024-03-23T13:18:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.319+0000 7f09067fc700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f08fc0a21e0 0x7f08fc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.319+0000 7f091545b700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f08fc0a21e0 0x7f08fc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.320+0000 7f09067fc700 1 -- 172.21.15.7:0/2945513975 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f09080ce0c0 con 0x7f0910106410 2024-03-23T13:18:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.320+0000 7f09067fc700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] conn(0x7f08fc0ab9c0 0x7f08fc0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.320+0000 7f09067fc700 1 -- 172.21.15.7:0/2945513975 --> [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08fc0ae490 con 0x7f08fc0ab9c0 2024-03-23T13:18:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.320+0000 7f09067fc700 1 -- 172.21.15.7:0/2945513975 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f09080ce430 con 0x7f0910106410 2024-03-23T13:18:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.320+0000 7f091545b700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f08fc0a21e0 0x7f08fc0a46a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f090c009c60 tx=0x7f090c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.320+0000 7f0914c5a700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] conn(0x7f08fc0ab9c0 0x7f08fc0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.320+0000 7f0914c5a700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] conn(0x7f08fc0ab9c0 0x7f08fc0adde0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.321+0000 7f09067fc700 1 -- 172.21.15.7:0/2945513975 <== osd.5 v2:172.21.15.155:6808/3878988779 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08fc0ae490 con 0x7f08fc0ab9c0 2024-03-23T13:18:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.337+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 --> [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f08f4002dc0 con 0x7f08fc0ab9c0 2024-03-23T13:18:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09067fc700 1 -- 172.21.15.7:0/2945513975 <== osd.5 v2:172.21.15.155:6808/3878988779 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f08f4002dc0 con 0x7f08fc0ab9c0 2024-03-23T13:18:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 >> [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] conn(0x7f08fc0ab9c0 msgr2=0x7f08fc0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09176bf700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] conn(0x7f08fc0ab9c0 0x7f08fc0adde0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f08fc0a21e0 msgr2=0x7f08fc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09176bf700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f08fc0a21e0 0x7f08fc0a46a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f090c009c60 tx=0x7f090c009260 comp rx=0 tx=0).stop 2024-03-23T13:18:15.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910106410 msgr2=0x7f0910103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09176bf700 1 --2- 172.21.15.7:0/2945513975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910106410 0x7f0910103850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f090800ef30 tx=0x7f090800c3a0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f0915c5c700 1 -- 172.21.15.7:0/2945513975 reap_dead start 2024-03-23T13:18:15.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 shutdown_connections 2024-03-23T13:18:15.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 >> 172.21.15.7:0/2945513975 conn(0x7f0910077150 msgr2=0x7f091006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:15.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 shutdown_connections 2024-03-23T13:18:15.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.338+0000 7f09176bf700 1 -- 172.21.15.7:0/2945513975 wait complete. 2024-03-23T13:18:15.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:15 smithi007 ceph-mon[32184]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:18:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.873+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2591898740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d0107540 msgr2=0x7f22d010f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.873+0000 7f22d4f2b700 1 --2- 172.21.15.7:0/2591898740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d0107540 0x7f22d010f110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f22c000b0a0 tx=0x7f22c000b3b0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.873+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2591898740 shutdown_connections 2024-03-23T13:18:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.873+0000 7f22d4f2b700 1 --2- 172.21.15.7:0/2591898740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d0107540 0x7f22d010f110 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.873+0000 7f22d4f2b700 1 --2- 172.21.15.7:0/2591898740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d0102460 0x7f22d0102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.873+0000 7f22d4f2b700 1 --2- 172.21.15.7:0/2591898740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d0102e10 0x7f22d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.873+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2591898740 >> 172.21.15.7:0/2591898740 conn(0x7f22d00fdcd0 msgr2=0x7f22d01000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.873+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2591898740 shutdown_connections 2024-03-23T13:18:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.873+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2591898740 wait complete. 2024-03-23T13:18:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.873+0000 7f22d4f2b700 1 Processor -- start 2024-03-23T13:18:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22d4f2b700 1 -- start start 2024-03-23T13:18:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22d4f2b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d0102460 0x7f22d019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22d4f2b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d0102e10 0x7f22d019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22d4f2b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d0107540 0x7f22d01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22d4f2b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0198e90 con 0x7f22d0102460 2024-03-23T13:18:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22d4f2b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0199000 con 0x7f22d0107540 2024-03-23T13:18:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22d4f2b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01991c0 con 0x7f22d0102e10 2024-03-23T13:18:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22ce59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d0102460 0x7f22d019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22ce59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d0102460 0x7f22d019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42226/0 (socket says 172.21.15.7:42226) 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22ce59c700 1 -- 172.21.15.7:0/2502417266 learned_addr learned my addr 172.21.15.7:0/2502417266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22cdd9b700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d0102e10 0x7f22d019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22ced9d700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d0107540 0x7f22d01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22ce59c700 1 -- 172.21.15.7:0/2502417266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d0102e10 msgr2=0x7f22d019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22ce59c700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d0102e10 0x7f22d019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22ce59c700 1 -- 172.21.15.7:0/2502417266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d0107540 msgr2=0x7f22d01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22ce59c700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22d0107540 0x7f22d01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22ce59c700 1 -- 172.21.15.7:0/2502417266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c0009d00 con 0x7f22d0102460 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22ce59c700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d0102460 0x7f22d019e610 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f22c800e9a0 tx=0x7f22c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22bf7fe700 1 -- 172.21.15.7:0/2502417266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8009790 con 0x7f22d0102460 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.874+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d01994a0 con 0x7f22d0102460 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.875+0000 7f22bf7fe700 1 -- 172.21.15.7:0/2502417266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c8010430 con 0x7f22d0102460 2024-03-23T13:18:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.875+0000 7f22bf7fe700 1 -- 172.21.15.7:0/2502417266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8018680 con 0x7f22d0102460 2024-03-23T13:18:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.875+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d01a55e0 con 0x7f22d0102460 2024-03-23T13:18:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.875+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22d0103940 con 0x7f22d0102460 2024-03-23T13:18:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.877+0000 7f22bf7fe700 1 -- 172.21.15.7:0/2502417266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f22c8010910 con 0x7f22d0102460 2024-03-23T13:18:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.877+0000 7f22bf7fe700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22b80a21e0 0x7f22b80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.877+0000 7f22cdd9b700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22b80a21e0 0x7f22b80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.877+0000 7f22bf7fe700 1 -- 172.21.15.7:0/2502417266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f22c8014070 con 0x7f22d0102460 2024-03-23T13:18:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.877+0000 7f22bf7fe700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] conn(0x7f22b80ab9c0 0x7f22b80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.878+0000 7f22bf7fe700 1 -- 172.21.15.7:0/2502417266 --> [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22b80ae490 con 0x7f22b80ab9c0 2024-03-23T13:18:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.878+0000 7f22bf7fe700 1 -- 172.21.15.7:0/2502417266 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f22c80ce950 con 0x7f22d0102460 2024-03-23T13:18:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.878+0000 7f22cdd9b700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22b80a21e0 0x7f22b80a46a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22d0199ea0 tx=0x7f22c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.878+0000 7f22ced9d700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] conn(0x7f22b80ab9c0 0x7f22b80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.878+0000 7f22ced9d700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] conn(0x7f22b80ab9c0 0x7f22b80adde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.879+0000 7f22bf7fe700 1 -- 172.21.15.7:0/2502417266 <== osd.4 v2:172.21.15.130:6808/568360270 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f22b80ae490 con 0x7f22b80ab9c0 2024-03-23T13:18:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.895+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 --> [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f22d01125b0 con 0x7f22b80ab9c0 2024-03-23T13:18:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.895+0000 7f22bf7fe700 1 -- 172.21.15.7:0/2502417266 <== osd.4 v2:172.21.15.130:6808/568360270 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f22d01125b0 con 0x7f22b80ab9c0 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 >> [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] conn(0x7f22b80ab9c0 msgr2=0x7f22b80adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22d4f2b700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] conn(0x7f22b80ab9c0 0x7f22b80adde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22b80a21e0 msgr2=0x7f22b80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22d4f2b700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f22b80a21e0 0x7f22b80a46a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22d0199ea0 tx=0x7f22c4009260 comp rx=0 tx=0).stop 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d0102460 msgr2=0x7f22d019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22d4f2b700 1 --2- 172.21.15.7:0/2502417266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d0102460 0x7f22d019e610 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f22c800e9a0 tx=0x7f22c800ecb0 comp rx=0 tx=0).stop 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22ced9d700 1 -- 172.21.15.7:0/2502417266 reap_dead start 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 shutdown_connections 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 >> 172.21.15.7:0/2502417266 conn(0x7f22d00fdcd0 msgr2=0x7f22d00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 shutdown_connections 2024-03-23T13:18:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:15.896+0000 7f22d4f2b700 1 -- 172.21.15.7:0/2502417266 wait complete. 2024-03-23T13:18:15.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:15 smithi130 ceph-mon[36225]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:18:16.037 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:15 smithi155 ceph-mon[36157]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:18:17.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.209+0000 7f4418d37700 1 -- 172.21.15.7:0/3894369414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102280 msgr2=0x7f4414102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.209+0000 7f4418d37700 1 --2- 172.21.15.7:0/3894369414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102280 0x7f4414102700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4408009990 tx=0x7f4408009ca0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.209+0000 7f4418d37700 1 -- 172.21.15.7:0/3894369414 shutdown_connections 2024-03-23T13:18:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.209+0000 7f4418d37700 1 --2- 172.21.15.7:0/3894369414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414102c40 0x7f441410f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.209+0000 7f4418d37700 1 --2- 172.21.15.7:0/3894369414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414108360 0x7f4414108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.209+0000 7f4418d37700 1 --2- 172.21.15.7:0/3894369414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102280 0x7f4414102700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.209+0000 7f4418d37700 1 -- 172.21.15.7:0/3894369414 >> 172.21.15.7:0/3894369414 conn(0x7f44140792c0 msgr2=0x7f44140796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.209+0000 7f4418d37700 1 -- 172.21.15.7:0/3894369414 shutdown_connections 2024-03-23T13:18:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.209+0000 7f4418d37700 1 -- 172.21.15.7:0/3894369414 wait complete. 2024-03-23T13:18:17.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4418d37700 1 Processor -- start 2024-03-23T13:18:17.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4418d37700 1 -- start start 2024-03-23T13:18:17.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4418d37700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414102280 0x7f44141986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:17.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4418d37700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102c40 0x7f4414198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:17.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4418d37700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414108360 0x7f441419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:17.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4418d37700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f441419d5a0 con 0x7f4414102c40 2024-03-23T13:18:17.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4418d37700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f441419d710 con 0x7f4414108360 2024-03-23T13:18:17.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4418d37700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f441419d8d0 con 0x7f4414102280 2024-03-23T13:18:17.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4411d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102c40 0x7f4414198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:17.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4411d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102c40 0x7f4414198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42242/0 (socket says 172.21.15.7:42242) 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.210+0000 7f4411d9b700 1 -- 172.21.15.7:0/923112309 learned_addr learned my addr 172.21.15.7:0/923112309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f4412d9d700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414108360 0x7f441419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f441259c700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414102280 0x7f44141986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f4411d9b700 1 -- 172.21.15.7:0/923112309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414102280 msgr2=0x7f44141986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f4411d9b700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414102280 0x7f44141986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f4411d9b700 1 -- 172.21.15.7:0/923112309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414108360 msgr2=0x7f441419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f4411d9b700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414108360 0x7f441419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f4411d9b700 1 -- 172.21.15.7:0/923112309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44080095f0 con 0x7f4414102c40 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f4412d9d700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414108360 0x7f441419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f4411d9b700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102c40 0x7f4414198c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4408004760 tx=0x7f4408004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f44037fe700 1 -- 172.21.15.7:0/923112309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440801d070 con 0x7f4414102c40 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441419db50 con 0x7f4414102c40 2024-03-23T13:18:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f44037fe700 1 -- 172.21.15.7:0/923112309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440800b5b0 con 0x7f4414102c40 2024-03-23T13:18:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f44037fe700 1 -- 172.21.15.7:0/923112309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44080173f0 con 0x7f4414102c40 2024-03-23T13:18:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.211+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141a5410 con 0x7f4414102c40 2024-03-23T13:18:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.213+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4414103940 con 0x7f4414102c40 2024-03-23T13:18:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.214+0000 7f44037fe700 1 -- 172.21.15.7:0/923112309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f4408017550 con 0x7f4414102c40 2024-03-23T13:18:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.214+0000 7f44037fe700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f43fc0a21e0 0x7f43fc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.215+0000 7f441259c700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f43fc0a21e0 0x7f43fc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.215+0000 7f44037fe700 1 -- 172.21.15.7:0/923112309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f44080cefe0 con 0x7f4414102c40 2024-03-23T13:18:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.215+0000 7f44037fe700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] conn(0x7f43fc0ab9c0 0x7f43fc0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.215+0000 7f44037fe700 1 -- 172.21.15.7:0/923112309 --> [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f43fc0ae490 con 0x7f43fc0ab9c0 2024-03-23T13:18:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.215+0000 7f44037fe700 1 -- 172.21.15.7:0/923112309 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f44080d15a0 con 0x7f4414102c40 2024-03-23T13:18:17.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.215+0000 7f4412d9d700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] conn(0x7f43fc0ab9c0 0x7f43fc0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:17.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.215+0000 7f441259c700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f43fc0a21e0 0x7f43fc0a46a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4404009be0 tx=0x7f4404009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:17.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.215+0000 7f4412d9d700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] conn(0x7f43fc0ab9c0 0x7f43fc0adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:17.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.216+0000 7f44037fe700 1 -- 172.21.15.7:0/923112309 <== osd.3 v2:172.21.15.7:6810/28573179 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f43fc0ae490 con 0x7f43fc0ab9c0 2024-03-23T13:18:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.233+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 --> [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4414112560 con 0x7f43fc0ab9c0 2024-03-23T13:18:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.233+0000 7f44037fe700 1 -- 172.21.15.7:0/923112309 <== osd.3 v2:172.21.15.7:6810/28573179 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4414112560 con 0x7f43fc0ab9c0 2024-03-23T13:18:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] conn(0x7f43fc0ab9c0 msgr2=0x7f43fc0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4418d37700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] conn(0x7f43fc0ab9c0 0x7f43fc0adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f43fc0a21e0 msgr2=0x7f43fc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4418d37700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f43fc0a21e0 0x7f43fc0a46a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4404009be0 tx=0x7f4404009260 comp rx=0 tx=0).stop 2024-03-23T13:18:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102c40 msgr2=0x7f4414198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4418d37700 1 --2- 172.21.15.7:0/923112309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102c40 0x7f4414198c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4408004760 tx=0x7f4408004790 comp rx=0 tx=0).stop 2024-03-23T13:18:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4412d9d700 1 -- 172.21.15.7:0/923112309 reap_dead start 2024-03-23T13:18:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 shutdown_connections 2024-03-23T13:18:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 >> 172.21.15.7:0/923112309 conn(0x7f44140792c0 msgr2=0x7f4414100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 shutdown_connections 2024-03-23T13:18:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.234+0000 7f4418d37700 1 -- 172.21.15.7:0/923112309 wait complete. 2024-03-23T13:18:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.620+0000 7fef9596b700 1 -- 172.21.15.7:0/400908341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef90101d30 msgr2=0x7fef90105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.620+0000 7fef9596b700 1 --2- 172.21.15.7:0/400908341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef90101d30 0x7fef90105d80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef84009990 tx=0x7fef84009ca0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.620+0000 7fef9596b700 1 -- 172.21.15.7:0/400908341 shutdown_connections 2024-03-23T13:18:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.620+0000 7fef9596b700 1 --2- 172.21.15.7:0/400908341 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef90106460 0x7fef9010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.620+0000 7fef9596b700 1 --2- 172.21.15.7:0/400908341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef90101380 0x7fef90101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.620+0000 7fef9596b700 1 --2- 172.21.15.7:0/400908341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef90101d30 0x7fef90105d80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.620+0000 7fef9596b700 1 -- 172.21.15.7:0/400908341 >> 172.21.15.7:0/400908341 conn(0x7fef90078160 msgr2=0x7fef90078570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.620+0000 7fef9596b700 1 -- 172.21.15.7:0/400908341 shutdown_connections 2024-03-23T13:18:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.620+0000 7fef9596b700 1 -- 172.21.15.7:0/400908341 wait complete. 2024-03-23T13:18:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef9596b700 1 Processor -- start 2024-03-23T13:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef9596b700 1 -- start start 2024-03-23T13:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef9596b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef90101380 0x7fef9019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef9596b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef90101d30 0x7fef9019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef9596b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef90106460 0x7fef901985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef9596b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fef90198c20 con 0x7fef90101380 2024-03-23T13:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef9596b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fef90198d60 con 0x7fef90106460 2024-03-23T13:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef9596b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef90198f20 con 0x7fef90101d30 2024-03-23T13:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef8effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef90101380 0x7fef9019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef8effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef90101380 0x7fef9019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42256/0 (socket says 172.21.15.7:42256) 2024-03-23T13:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef8effd700 1 -- 172.21.15.7:0/797861800 learned_addr learned my addr 172.21.15.7:0/797861800 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef8e7fc700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef90101d30 0x7fef9019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.621+0000 7fef8f7fe700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef90106460 0x7fef901985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef8effd700 1 -- 172.21.15.7:0/797861800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef90101d30 msgr2=0x7fef9019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef8effd700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef90101d30 0x7fef9019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef8effd700 1 -- 172.21.15.7:0/797861800 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef90106460 msgr2=0x7fef901985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef8effd700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef90106460 0x7fef901985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef8effd700 1 -- 172.21.15.7:0/797861800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef840095f0 con 0x7fef90101380 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef8effd700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef90101380 0x7fef9019e400 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef8000e9a0 tx=0x7fef8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef94969700 1 -- 172.21.15.7:0/797861800 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80009790 con 0x7fef90101380 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef94969700 1 -- 172.21.15.7:0/797861800 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef80010430 con 0x7fef90101380 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef90199200 con 0x7fef90101380 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef94969700 1 -- 172.21.15.7:0/797861800 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80018680 con 0x7fef90101380 2024-03-23T13:18:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.622+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef901a54a0 con 0x7fef90101380 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.623+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef90111460 con 0x7fef90101380 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.624+0000 7fef94969700 1 -- 172.21.15.7:0/797861800 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fef80010910 con 0x7fef90101380 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.625+0000 7fef94969700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef780a2260 0x7fef780a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.625+0000 7fef8e7fc700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef780a2260 0x7fef780a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.625+0000 7fef94969700 1 -- 172.21.15.7:0/797861800 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fef80014070 con 0x7fef90101380 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.625+0000 7fef94969700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] conn(0x7fef780aba40 0x7fef780ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.625+0000 7fef94969700 1 -- 172.21.15.7:0/797861800 --> [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef780ae510 con 0x7fef780aba40 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.625+0000 7fef94969700 1 -- 172.21.15.7:0/797861800 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fef800ce8d0 con 0x7fef90101380 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.625+0000 7fef8f7fe700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] conn(0x7fef780aba40 0x7fef780ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.625+0000 7fef8e7fc700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef780a2260 0x7fef780a4720 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef84006010 tx=0x7fef84005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.626+0000 7fef8f7fe700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] conn(0x7fef780aba40 0x7fef780ade60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.627+0000 7fef94969700 1 -- 172.21.15.7:0/797861800 <== osd.10 v2:172.21.15.130:6824/1468057097 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef780ae510 con 0x7fef780aba40 2024-03-23T13:18:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.646+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 --> [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef900628c0 con 0x7fef780aba40 2024-03-23T13:18:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.646+0000 7fef94969700 1 -- 172.21.15.7:0/797861800 <== osd.10 v2:172.21.15.130:6824/1468057097 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fef900628c0 con 0x7fef780aba40 2024-03-23T13:18:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 >> [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] conn(0x7fef780aba40 msgr2=0x7fef780ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef9596b700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] conn(0x7fef780aba40 0x7fef780ade60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef780a2260 msgr2=0x7fef780a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef9596b700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef780a2260 0x7fef780a4720 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef84006010 tx=0x7fef84005960 comp rx=0 tx=0).stop 2024-03-23T13:18:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef90101380 msgr2=0x7fef9019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef9596b700 1 --2- 172.21.15.7:0/797861800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef90101380 0x7fef9019e400 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef8000e9a0 tx=0x7fef8000ecb0 comp rx=0 tx=0).stop 2024-03-23T13:18:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef8f7fe700 1 -- 172.21.15.7:0/797861800 reap_dead start 2024-03-23T13:18:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 shutdown_connections 2024-03-23T13:18:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 >> 172.21.15.7:0/797861800 conn(0x7fef90078160 msgr2=0x7fef9006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 shutdown_connections 2024-03-23T13:18:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:17.647+0000 7fef9596b700 1 -- 172.21.15.7:0/797861800 wait complete. 2024-03-23T13:18:17.878 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:17 smithi007 ceph-mon[32184]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T13:18:17.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:17 smithi130 ceph-mon[36225]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T13:18:18.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:17 smithi155 ceph-mon[36157]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T13:18:19.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:19 smithi007 ceph-mon[32184]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T13:18:19.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:19 smithi130 ceph-mon[36225]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T13:18:20.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:19 smithi155 ceph-mon[36157]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T13:18:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 -- 172.21.15.7:0/2801703793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c108230 msgr2=0x7f562c108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 --2- 172.21.15.7:0/2801703793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c108230 0x7f562c108610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f56280099c0 tx=0x7f5628009cd0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 -- 172.21.15.7:0/2801703793 shutdown_connections 2024-03-23T13:18:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 --2- 172.21.15.7:0/2801703793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f562c102c10 0x7f562c10f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 --2- 172.21.15.7:0/2801703793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c108230 0x7f562c108610 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 --2- 172.21.15.7:0/2801703793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f562c102250 0x7f562c1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 -- 172.21.15.7:0/2801703793 >> 172.21.15.7:0/2801703793 conn(0x7f562c079330 msgr2=0x7f562c079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 -- 172.21.15.7:0/2801703793 shutdown_connections 2024-03-23T13:18:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 -- 172.21.15.7:0/2801703793 wait complete. 2024-03-23T13:18:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 Processor -- start 2024-03-23T13:18:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.303+0000 7f5633d6a700 1 -- start start 2024-03-23T13:18:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5633d6a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f562c102250 0x7f562c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5633d6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c102c10 0x7f562c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5633d6a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f562c108230 0x7f562c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5633d6a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f562c19d630 con 0x7f562c102c10 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5633d6a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f562c19d7a0 con 0x7f562c102250 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5633d6a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f562c19d960 con 0x7f562c108230 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5631305700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c102c10 0x7f562c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5631305700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c102c10 0x7f562c198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42294/0 (socket says 172.21.15.7:42294) 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5631305700 1 -- 172.21.15.7:0/527216475 learned_addr learned my addr 172.21.15.7:0/527216475 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5631b06700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f562c102250 0x7f562c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5632307700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f562c108230 0x7f562c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5631305700 1 -- 172.21.15.7:0/527216475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f562c108230 msgr2=0x7f562c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5631305700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f562c108230 0x7f562c19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5631305700 1 -- 172.21.15.7:0/527216475 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f562c102250 msgr2=0x7f562c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5631305700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f562c102250 0x7f562c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5631305700 1 -- 172.21.15.7:0/527216475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56280095f0 con 0x7f562c102c10 2024-03-23T13:18:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5631305700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c102c10 0x7f562c198c90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f561c00e9a0 tx=0x7f561c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5622ffd700 1 -- 172.21.15.7:0/527216475 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c009790 con 0x7f562c102c10 2024-03-23T13:18:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562c19dc40 con 0x7f562c102c10 2024-03-23T13:18:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5622ffd700 1 -- 172.21.15.7:0/527216475 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561c010430 con 0x7f562c102c10 2024-03-23T13:18:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5622ffd700 1 -- 172.21.15.7:0/527216475 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c018680 con 0x7f562c102c10 2024-03-23T13:18:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.304+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562c1a54a0 con 0x7f562c102c10 2024-03-23T13:18:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.305+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f562c112590 con 0x7f562c102c10 2024-03-23T13:18:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.307+0000 7f5622ffd700 1 -- 172.21.15.7:0/527216475 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f561c010910 con 0x7f562c102c10 2024-03-23T13:18:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.307+0000 7f5622ffd700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f56180a21e0 0x7f56180a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.307+0000 7f5631b06700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f56180a21e0 0x7f56180a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.307+0000 7f5622ffd700 1 -- 172.21.15.7:0/527216475 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f561c014070 con 0x7f562c102c10 2024-03-23T13:18:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.308+0000 7f5622ffd700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] conn(0x7f56180ab9c0 0x7f56180adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.308+0000 7f5622ffd700 1 -- 172.21.15.7:0/527216475 --> [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56180ae490 con 0x7f56180ab9c0 2024-03-23T13:18:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.308+0000 7f5622ffd700 1 -- 172.21.15.7:0/527216475 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f561c0ced10 con 0x7f562c102c10 2024-03-23T13:18:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.308+0000 7f5632307700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] conn(0x7f56180ab9c0 0x7f56180adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.308+0000 7f5631b06700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f56180a21e0 0x7f56180a46a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5628000c00 tx=0x7f5628005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.308+0000 7f5632307700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] conn(0x7f56180ab9c0 0x7f56180adde0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.309+0000 7f5622ffd700 1 -- 172.21.15.7:0/527216475 <== osd.9 v2:172.21.15.7:6826/439190456 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f56180ae490 con 0x7f56180ab9c0 2024-03-23T13:18:20.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.325+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 --> [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f562c062890 con 0x7f56180ab9c0 2024-03-23T13:18:20.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.325+0000 7f5622ffd700 1 -- 172.21.15.7:0/527216475 <== osd.9 v2:172.21.15.7:6826/439190456 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f562c062890 con 0x7f56180ab9c0 2024-03-23T13:18:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] conn(0x7f56180ab9c0 msgr2=0x7f56180adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5633d6a700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] conn(0x7f56180ab9c0 0x7f56180adde0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f56180a21e0 msgr2=0x7f56180a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5633d6a700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f56180a21e0 0x7f56180a46a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5628000c00 tx=0x7f5628005fb0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c102c10 msgr2=0x7f562c198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5633d6a700 1 --2- 172.21.15.7:0/527216475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c102c10 0x7f562c198c90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f561c00e9a0 tx=0x7f561c00ecb0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5632307700 1 -- 172.21.15.7:0/527216475 reap_dead start 2024-03-23T13:18:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 shutdown_connections 2024-03-23T13:18:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 >> 172.21.15.7:0/527216475 conn(0x7f562c079330 msgr2=0x7f562c100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 shutdown_connections 2024-03-23T13:18:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.326+0000 7f5633d6a700 1 -- 172.21.15.7:0/527216475 wait complete. 2024-03-23T13:18:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.933+0000 7fce81472700 1 -- 172.21.15.7:0/3925396859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce7c108250 msgr2=0x7fce7c108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.933+0000 7fce81472700 1 --2- 172.21.15.7:0/3925396859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce7c108250 0x7fce7c108630 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099c0 tx=0x7fce6c009cd0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.933+0000 7fce81472700 1 -- 172.21.15.7:0/3925396859 shutdown_connections 2024-03-23T13:18:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.933+0000 7fce81472700 1 --2- 172.21.15.7:0/3925396859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce7c06a850 0x7fce7c10f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.933+0000 7fce81472700 1 --2- 172.21.15.7:0/3925396859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce7c108250 0x7fce7c108630 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.933+0000 7fce81472700 1 --2- 172.21.15.7:0/3925396859 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce7c069eb0 0x7fce7c06a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.933+0000 7fce81472700 1 -- 172.21.15.7:0/3925396859 >> 172.21.15.7:0/3925396859 conn(0x7fce7c078030 msgr2=0x7fce7c078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.933+0000 7fce81472700 1 -- 172.21.15.7:0/3925396859 shutdown_connections 2024-03-23T13:18:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.933+0000 7fce81472700 1 -- 172.21.15.7:0/3925396859 wait complete. 2024-03-23T13:18:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.933+0000 7fce81472700 1 Processor -- start 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce81472700 1 -- start start 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce81472700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce7c069eb0 0x7fce7c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce81472700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce7c06a850 0x7fce7c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce81472700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce7c108250 0x7fce7c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce81472700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c19d580 con 0x7fce7c069eb0 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce81472700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c19d6f0 con 0x7fce7c06a850 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce81472700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c19d8b0 con 0x7fce7c108250 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce7c069eb0 0x7fce7c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce7c069eb0 0x7fce7c1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42312/0 (socket says 172.21.15.7:42312) 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7affd700 1 -- 172.21.15.7:0/2932700581 learned_addr learned my addr 172.21.15.7:0/2932700581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7a7fc700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce7c06a850 0x7fce7c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7b7fe700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce7c108250 0x7fce7c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7affd700 1 -- 172.21.15.7:0/2932700581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce7c108250 msgr2=0x7fce7c19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7affd700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce7c108250 0x7fce7c19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7affd700 1 -- 172.21.15.7:0/2932700581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce7c06a850 msgr2=0x7fce7c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7affd700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce7c06a850 0x7fce7c198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7affd700 1 -- 172.21.15.7:0/2932700581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6c0095f0 con 0x7fce7c069eb0 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce7affd700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce7c069eb0 0x7fce7c1986a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fce6c005910 tx=0x7fce6c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.934+0000 7fce63fff700 1 -- 172.21.15.7:0/2932700581 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c01d070 con 0x7fce7c069eb0 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.935+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7c19db30 con 0x7fce7c069eb0 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.935+0000 7fce63fff700 1 -- 172.21.15.7:0/2932700581 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce6c003680 con 0x7fce7c069eb0 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.935+0000 7fce63fff700 1 -- 172.21.15.7:0/2932700581 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0173f0 con 0x7fce7c069eb0 2024-03-23T13:18:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.935+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce7c1a53f0 con 0x7fce7c069eb0 2024-03-23T13:18:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.935+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce7c103900 con 0x7fce7c069eb0 2024-03-23T13:18:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.937+0000 7fce63fff700 1 -- 172.21.15.7:0/2932700581 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fce6c022600 con 0x7fce7c069eb0 2024-03-23T13:18:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.937+0000 7fce63fff700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce640a21e0 0x7fce640a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.937+0000 7fce7a7fc700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce640a21e0 0x7fce640a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.937+0000 7fce63fff700 1 -- 172.21.15.7:0/2932700581 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fce6c0ce960 con 0x7fce7c069eb0 2024-03-23T13:18:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.937+0000 7fce63fff700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] conn(0x7fce640ab9c0 0x7fce640adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.937+0000 7fce63fff700 1 -- 172.21.15.7:0/2932700581 --> [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce640ae490 con 0x7fce640ab9c0 2024-03-23T13:18:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.937+0000 7fce63fff700 1 -- 172.21.15.7:0/2932700581 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fce6c0cee00 con 0x7fce7c069eb0 2024-03-23T13:18:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.938+0000 7fce7b7fe700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] conn(0x7fce640ab9c0 0x7fce640adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.938+0000 7fce7a7fc700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce640a21e0 0x7fce640a46a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0785a0 tx=0x7fce70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.938+0000 7fce7b7fe700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] conn(0x7fce640ab9c0 0x7fce640adde0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.939+0000 7fce63fff700 1 -- 172.21.15.7:0/2932700581 <== osd.6 v2:172.21.15.7:6818/2961397524 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce640ae490 con 0x7fce640ab9c0 2024-03-23T13:18:20.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.955+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 --> [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce7c112520 con 0x7fce640ab9c0 2024-03-23T13:18:20.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.955+0000 7fce63fff700 1 -- 172.21.15.7:0/2932700581 <== osd.6 v2:172.21.15.7:6818/2961397524 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fce7c112520 con 0x7fce640ab9c0 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] conn(0x7fce640ab9c0 msgr2=0x7fce640adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce81472700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] conn(0x7fce640ab9c0 0x7fce640adde0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce640a21e0 msgr2=0x7fce640a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce81472700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fce640a21e0 0x7fce640a46a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0785a0 tx=0x7fce70009260 comp rx=0 tx=0).stop 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce7c069eb0 msgr2=0x7fce7c1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce81472700 1 --2- 172.21.15.7:0/2932700581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce7c069eb0 0x7fce7c1986a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fce6c005910 tx=0x7fce6c00f690 comp rx=0 tx=0).stop 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce7b7fe700 1 -- 172.21.15.7:0/2932700581 reap_dead start 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 shutdown_connections 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 >> 172.21.15.7:0/2932700581 conn(0x7fce7c078030 msgr2=0x7fce7c0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 shutdown_connections 2024-03-23T13:18:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:20.956+0000 7fce81472700 1 -- 172.21.15.7:0/2932700581 wait complete. 2024-03-23T13:18:21.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:21 smithi007 ceph-mon[32184]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:21.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:21 smithi130 ceph-mon[36225]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:22.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:21 smithi155 ceph-mon[36157]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:22.076 INFO:teuthology.orchestra.run.smithi007.stdout:163208757298 2024-03-23T13:18:22.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-23T13:18:22.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.377+0000 7f03f3c89700 1 -- 172.21.15.7:0/164331338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03ec1011b0 msgr2=0x7f03ec10f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.377+0000 7f03f3c89700 1 --2- 172.21.15.7:0/164331338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03ec1011b0 0x7f03ec10f110 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f03e400b0a0 tx=0x7f03e400b3b0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 -- 172.21.15.7:0/164331338 shutdown_connections 2024-03-23T13:18:22.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 --2- 172.21.15.7:0/164331338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03ec1011b0 0x7f03ec10f110 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 --2- 172.21.15.7:0/164331338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ec107270 0x7f03ec107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 --2- 172.21.15.7:0/164331338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03ec1007f0 0x7f03ec100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 -- 172.21.15.7:0/164331338 >> 172.21.15.7:0/164331338 conn(0x7f03ec077700 msgr2=0x7f03ec0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:22.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 -- 172.21.15.7:0/164331338 shutdown_connections 2024-03-23T13:18:22.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 -- 172.21.15.7:0/164331338 wait complete. 2024-03-23T13:18:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 Processor -- start 2024-03-23T13:18:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 -- start start 2024-03-23T13:18:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ec1007f0 0x7f03ec1019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03ec1011b0 0x7f03ec101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03ec107270 0x7f03ec102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.378+0000 7f03f3c89700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec1a6d70 con 0x7f03ec107270 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f3c89700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec1a6ee0 con 0x7f03ec1011b0 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f3c89700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec1a70a0 con 0x7f03ec1007f0 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f2226700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03ec107270 0x7f03ec102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f2226700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03ec107270 0x7f03ec102450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42356/0 (socket says 172.21.15.7:42356) 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f1224700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03ec1011b0 0x7f03ec101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f2226700 1 -- 172.21.15.7:0/1660781646 learned_addr learned my addr 172.21.15.7:0/1660781646 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f1a25700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ec1007f0 0x7f03ec1019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f2226700 1 -- 172.21.15.7:0/1660781646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ec1007f0 msgr2=0x7f03ec1019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f2226700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03ec1007f0 0x7f03ec1019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f2226700 1 -- 172.21.15.7:0/1660781646 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03ec1011b0 msgr2=0x7f03ec101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f2226700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03ec1011b0 0x7f03ec101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f2226700 1 -- 172.21.15.7:0/1660781646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e4009d00 con 0x7f03ec107270 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f1224700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03ec1011b0 0x7f03ec101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f2226700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03ec107270 0x7f03ec102450 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f03e4006070 tx=0x7f03e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03deffd700 1 -- 172.21.15.7:0/1660781646 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e400e040 con 0x7f03ec107270 2024-03-23T13:18:22.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ec1a7320 con 0x7f03ec107270 2024-03-23T13:18:22.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03deffd700 1 -- 172.21.15.7:0/1660781646 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03e401d470 con 0x7f03ec107270 2024-03-23T13:18:22.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.379+0000 7f03deffd700 1 -- 172.21.15.7:0/1660781646 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e401b6a0 con 0x7f03ec107270 2024-03-23T13:18:22.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.380+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ec1a7790 con 0x7f03ec107270 2024-03-23T13:18:22.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.380+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03ec1126d0 con 0x7f03ec107270 2024-03-23T13:18:22.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.382+0000 7f03deffd700 1 -- 172.21.15.7:0/1660781646 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f03e401d990 con 0x7f03ec107270 2024-03-23T13:18:22.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.382+0000 7f03deffd700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f03d80a2260 0x7f03d80a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:22.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.383+0000 7f03f1a25700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f03d80a2260 0x7f03d80a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:22.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.383+0000 7f03deffd700 1 -- 172.21.15.7:0/1660781646 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f03e40ce980 con 0x7f03ec107270 2024-03-23T13:18:22.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.383+0000 7f03deffd700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] conn(0x7f03d80aba40 0x7f03d80ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:22.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.383+0000 7f03deffd700 1 -- 172.21.15.7:0/1660781646 --> [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03d80ae510 con 0x7f03d80aba40 2024-03-23T13:18:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.383+0000 7f03deffd700 1 -- 172.21.15.7:0/1660781646 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f03e40d32c0 con 0x7f03ec107270 2024-03-23T13:18:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.383+0000 7f03f1224700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] conn(0x7f03d80aba40 0x7f03d80ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.383+0000 7f03f1a25700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f03d80a2260 0x7f03d80a4720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009be0 tx=0x7f03e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.383+0000 7f03f1224700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] conn(0x7f03d80aba40 0x7f03d80ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:22.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.384+0000 7f03deffd700 1 -- 172.21.15.7:0/1660781646 <== osd.7 v2:172.21.15.130:6816/869404947 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03d80ae510 con 0x7f03d80aba40 2024-03-23T13:18:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.405+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 --> [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03ec0628c0 con 0x7f03d80aba40 2024-03-23T13:18:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03deffd700 1 -- 172.21.15.7:0/1660781646 <== osd.7 v2:172.21.15.130:6816/869404947 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f03ec0628c0 con 0x7f03d80aba40 2024-03-23T13:18:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 >> [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] conn(0x7f03d80aba40 msgr2=0x7f03d80ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f3c89700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] conn(0x7f03d80aba40 0x7f03d80ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f03d80a2260 msgr2=0x7f03d80a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f3c89700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f03d80a2260 0x7f03d80a4720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009be0 tx=0x7f03e8009260 comp rx=0 tx=0).stop 2024-03-23T13:18:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03ec107270 msgr2=0x7f03ec102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f3c89700 1 --2- 172.21.15.7:0/1660781646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03ec107270 0x7f03ec102450 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f03e4006070 tx=0x7f03e400be20 comp rx=0 tx=0).stop 2024-03-23T13:18:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f2226700 1 -- 172.21.15.7:0/1660781646 reap_dead start 2024-03-23T13:18:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 shutdown_connections 2024-03-23T13:18:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 >> 172.21.15.7:0/1660781646 conn(0x7f03ec077700 msgr2=0x7f03ec1051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 shutdown_connections 2024-03-23T13:18:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:22.406+0000 7f03f3c89700 1 -- 172.21.15.7:0/1660781646 wait complete. 2024-03-23T13:18:22.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.261+0000 7f9aad336700 1 -- 172.21.15.7:0/1549093427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa8101aa0 msgr2=0x7f9aa810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.261+0000 7f9aad336700 1 --2- 172.21.15.7:0/1549093427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa8101aa0 0x7f9aa810df40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000b0a0 tx=0x7f9aa000b3b0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.261+0000 7f9aad336700 1 -- 172.21.15.7:0/1549093427 shutdown_connections 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.261+0000 7f9aad336700 1 --2- 172.21.15.7:0/1549093427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa8101aa0 0x7f9aa810df40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.261+0000 7f9aad336700 1 --2- 172.21.15.7:0/1549093427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9aa8107190 0x7f9aa8107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.261+0000 7f9aad336700 1 --2- 172.21.15.7:0/1549093427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aa81010e0 0x7f9aa8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.261+0000 7f9aad336700 1 -- 172.21.15.7:0/1549093427 >> 172.21.15.7:0/1549093427 conn(0x7f9aa8078110 msgr2=0x7f9aa8078520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.261+0000 7f9aad336700 1 -- 172.21.15.7:0/1549093427 shutdown_connections 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.261+0000 7f9aad336700 1 -- 172.21.15.7:0/1549093427 wait complete. 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aad336700 1 Processor -- start 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aad336700 1 -- start start 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aad336700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aa81010e0 0x7f9aa8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aad336700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa8101aa0 0x7f9aa8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aad336700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9aa8107190 0x7f9aa819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aad336700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa819d620 con 0x7f9aa8101aa0 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aad336700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa819d790 con 0x7f9aa8107190 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aad336700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa819d950 con 0x7f9aa81010e0 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa8101aa0 0x7f9aa8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aa81010e0 0x7f9aa8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aa81010e0 0x7f9aa8198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:34472/0 (socket says 172.21.15.7:34472) 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa8101aa0 0x7f9aa8198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42286/0 (socket says 172.21.15.7:42286) 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa6ffd700 1 -- 172.21.15.7:0/1089488336 learned_addr learned my addr 172.21.15.7:0/1089488336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa77fe700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9aa8107190 0x7f9aa819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa6ffd700 1 -- 172.21.15.7:0/1089488336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9aa8107190 msgr2=0x7f9aa819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa6ffd700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9aa8107190 0x7f9aa819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa6ffd700 1 -- 172.21.15.7:0/1089488336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa8101aa0 msgr2=0x7f9aa8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa6ffd700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa8101aa0 0x7f9aa8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa6ffd700 1 -- 172.21.15.7:0/1089488336 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa0009d00 con 0x7f9aa81010e0 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa67fc700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa8101aa0 0x7f9aa8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.262+0000 7f9aa77fe700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9aa8107190 0x7f9aa819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.263+0000 7f9aa6ffd700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aa81010e0 0x7f9aa8198740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800e9a0 tx=0x7f9a9800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.263+0000 7f9a8ffff700 1 -- 172.21.15.7:0/1089488336 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98009790 con 0x7f9aa81010e0 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.263+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa819dc30 con 0x7f9aa81010e0 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.263+0000 7f9a8ffff700 1 -- 172.21.15.7:0/1089488336 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a98010430 con 0x7f9aa81010e0 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.263+0000 7f9a8ffff700 1 -- 172.21.15.7:0/1089488336 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98018680 con 0x7f9aa81010e0 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.263+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa81a5490 con 0x7f9aa81010e0 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.264+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9aa81027d0 con 0x7f9aa81010e0 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.265+0000 7f9a8ffff700 1 -- 172.21.15.7:0/1089488336 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f9a98010910 con 0x7f9aa81010e0 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.265+0000 7f9a8ffff700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a900a21e0 0x7f9a900a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.266+0000 7f9aa67fc700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a900a21e0 0x7f9a900a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.266+0000 7f9a8ffff700 1 -- 172.21.15.7:0/1089488336 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9a98014070 con 0x7f9aa81010e0 2024-03-23T13:18:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.266+0000 7f9a8ffff700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] conn(0x7f9a900ab9c0 0x7f9a900adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.266+0000 7f9a8ffff700 1 -- 172.21.15.7:0/1089488336 --> [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a900ae490 con 0x7f9a900ab9c0 2024-03-23T13:18:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.266+0000 7f9a8ffff700 1 -- 172.21.15.7:0/1089488336 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f9a980ce920 con 0x7f9aa81010e0 2024-03-23T13:18:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.266+0000 7f9aa67fc700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a900a21e0 0x7f9a900a46a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9aa806ba00 tx=0x7f9a9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.266+0000 7f9aa77fe700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] conn(0x7f9a900ab9c0 0x7f9a900adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.266+0000 7f9aa77fe700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] conn(0x7f9a900ab9c0 0x7f9a900adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.267+0000 7f9a8ffff700 1 -- 172.21.15.7:0/1089488336 <== osd.1 v2:172.21.15.130:6800/704785998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a900ae490 con 0x7f9a900ab9c0 2024-03-23T13:18:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.284+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 --> [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9aa8111430 con 0x7f9a900ab9c0 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.285+0000 7f9a8ffff700 1 -- 172.21.15.7:0/1089488336 <== osd.1 v2:172.21.15.130:6800/704785998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9aa8111430 con 0x7f9a900ab9c0 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.285+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 >> [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] conn(0x7f9a900ab9c0 msgr2=0x7f9a900adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.285+0000 7f9aad336700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] conn(0x7f9a900ab9c0 0x7f9a900adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.285+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a900a21e0 msgr2=0x7f9a900a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.285+0000 7f9aad336700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a900a21e0 0x7f9a900a46a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9aa806ba00 tx=0x7f9a9c009260 comp rx=0 tx=0).stop 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.285+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aa81010e0 msgr2=0x7f9aa8198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.285+0000 7f9aad336700 1 --2- 172.21.15.7:0/1089488336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aa81010e0 0x7f9aa8198740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800e9a0 tx=0x7f9a9800ecb0 comp rx=0 tx=0).stop 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.285+0000 7f9aa77fe700 1 -- 172.21.15.7:0/1089488336 reap_dead start 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.286+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 shutdown_connections 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.286+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 >> 172.21.15.7:0/1089488336 conn(0x7f9aa8078110 msgr2=0x7f9aa81053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.286+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 shutdown_connections 2024-03-23T13:18:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:18.286+0000 7f9aad336700 1 -- 172.21.15.7:0/1089488336 wait complete. 2024-03-23T13:18:23.525 INFO:teuthology.orchestra.run.smithi007.stdout:55834574919 2024-03-23T13:18:23.526 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-23T13:18:23.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:23 smithi007 ceph-mon[32184]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-03-23T13:18:23.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:23 smithi130 ceph-mon[36225]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-03-23T13:18:23.957 INFO:teuthology.orchestra.run.smithi007.stdout:107374182460 2024-03-23T13:18:23.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-23T13:18:24.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:23 smithi155 ceph-mon[36157]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-03-23T13:18:24.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.134+0000 7fdb83f9d700 1 -- 172.21.15.7:0/1505649806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb7c06a850 msgr2=0x7fdb7c10f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.134+0000 7fdb83f9d700 1 --2- 172.21.15.7:0/1505649806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb7c06a850 0x7fdb7c10f090 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400b0d0 tx=0x7fdb7400b3e0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.134+0000 7fdb83f9d700 1 -- 172.21.15.7:0/1505649806 shutdown_connections 2024-03-23T13:18:24.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.134+0000 7fdb83f9d700 1 --2- 172.21.15.7:0/1505649806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb7c06a850 0x7fdb7c10f090 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.134+0000 7fdb83f9d700 1 --2- 172.21.15.7:0/1505649806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb7c108250 0x7fdb7c108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.134+0000 7fdb83f9d700 1 --2- 172.21.15.7:0/1505649806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb7c069eb0 0x7fdb7c06a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.134+0000 7fdb83f9d700 1 -- 172.21.15.7:0/1505649806 >> 172.21.15.7:0/1505649806 conn(0x7fdb7c078030 msgr2=0x7fdb7c078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.134+0000 7fdb83f9d700 1 -- 172.21.15.7:0/1505649806 shutdown_connections 2024-03-23T13:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.134+0000 7fdb83f9d700 1 -- 172.21.15.7:0/1505649806 wait complete. 2024-03-23T13:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb83f9d700 1 Processor -- start 2024-03-23T13:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb83f9d700 1 -- start start 2024-03-23T13:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb83f9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb7c069eb0 0x7fdb7c1011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb83f9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb7c06a850 0x7fdb7c101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb83f9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb7c108250 0x7fdb7c101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb83f9d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c1a6c60 con 0x7fdb7c06a850 2024-03-23T13:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb83f9d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c1a6dd0 con 0x7fdb7c069eb0 2024-03-23T13:18:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb83f9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c1a6f90 con 0x7fdb7c108250 2024-03-23T13:18:24.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb81538700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb7c06a850 0x7fdb7c101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:24.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb81538700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb7c06a850 0x7fdb7c101710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42376/0 (socket says 172.21.15.7:42376) 2024-03-23T13:18:24.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.135+0000 7fdb81538700 1 -- 172.21.15.7:0/471299768 learned_addr learned my addr 172.21.15.7:0/471299768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb81d39700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb7c069eb0 0x7fdb7c1011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb8253a700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb7c108250 0x7fdb7c101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb81538700 1 -- 172.21.15.7:0/471299768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb7c108250 msgr2=0x7fdb7c101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb81538700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb7c108250 0x7fdb7c101c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb81538700 1 -- 172.21.15.7:0/471299768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb7c069eb0 msgr2=0x7fdb7c1011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb81538700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb7c069eb0 0x7fdb7c1011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb81538700 1 -- 172.21.15.7:0/471299768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb74009d00 con 0x7fdb7c06a850 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb81d39700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb7c069eb0 0x7fdb7c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb81538700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb7c06a850 0x7fdb7c101710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00e9a0 tx=0x7fdb6c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb72ffd700 1 -- 172.21.15.7:0/471299768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c009790 con 0x7fdb7c06a850 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7c1a7270 con 0x7fdb7c06a850 2024-03-23T13:18:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb72ffd700 1 -- 172.21.15.7:0/471299768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6c00cbd0 con 0x7fdb7c06a850 2024-03-23T13:18:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb72ffd700 1 -- 172.21.15.7:0/471299768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c018680 con 0x7fdb7c06a850 2024-03-23T13:18:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.136+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7c1a77c0 con 0x7fdb7c06a850 2024-03-23T13:18:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.137+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb7c1124b0 con 0x7fdb7c06a850 2024-03-23T13:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.138+0000 7fdb72ffd700 1 -- 172.21.15.7:0/471299768 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fdb6c00cd40 con 0x7fdb7c06a850 2024-03-23T13:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.139+0000 7fdb72ffd700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdb680a2260 0x7fdb680a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.139+0000 7fdb81d39700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdb680a2260 0x7fdb680a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.139+0000 7fdb72ffd700 1 -- 172.21.15.7:0/471299768 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fdb6c014070 con 0x7fdb7c06a850 2024-03-23T13:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.139+0000 7fdb72ffd700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] conn(0x7fdb680aba40 0x7fdb680ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.139+0000 7fdb72ffd700 1 -- 172.21.15.7:0/471299768 --> [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb680ae510 con 0x7fdb680aba40 2024-03-23T13:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.139+0000 7fdb72ffd700 1 -- 172.21.15.7:0/471299768 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fdb6c0cea30 con 0x7fdb7c06a850 2024-03-23T13:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.139+0000 7fdb8253a700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] conn(0x7fdb680aba40 0x7fdb680ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.140+0000 7fdb81d39700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdb680a2260 0x7fdb680a4720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009c90 tx=0x7fdb78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.140+0000 7fdb8253a700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] conn(0x7fdb680aba40 0x7fdb680ade60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.141+0000 7fdb72ffd700 1 -- 172.21.15.7:0/471299768 <== osd.8 v2:172.21.15.155:6816/2565353660 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb680ae510 con 0x7fdb680aba40 2024-03-23T13:18:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.157+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 --> [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb7c0628c0 con 0x7fdb680aba40 2024-03-23T13:18:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb72ffd700 1 -- 172.21.15.7:0/471299768 <== osd.8 v2:172.21.15.155:6816/2565353660 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdb7c0628c0 con 0x7fdb680aba40 2024-03-23T13:18:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 >> [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] conn(0x7fdb680aba40 msgr2=0x7fdb680ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb83f9d700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] conn(0x7fdb680aba40 0x7fdb680ade60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdb680a2260 msgr2=0x7fdb680a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb83f9d700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fdb680a2260 0x7fdb680a4720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009c90 tx=0x7fdb78009260 comp rx=0 tx=0).stop 2024-03-23T13:18:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb7c06a850 msgr2=0x7fdb7c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb83f9d700 1 --2- 172.21.15.7:0/471299768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb7c06a850 0x7fdb7c101710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00e9a0 tx=0x7fdb6c00ed60 comp rx=0 tx=0).stop 2024-03-23T13:18:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb8253a700 1 -- 172.21.15.7:0/471299768 reap_dead start 2024-03-23T13:18:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 shutdown_connections 2024-03-23T13:18:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 >> 172.21.15.7:0/471299768 conn(0x7fdb7c078030 msgr2=0x7fdb7c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 shutdown_connections 2024-03-23T13:18:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.158+0000 7fdb83f9d700 1 -- 172.21.15.7:0/471299768 wait complete. 2024-03-23T13:18:24.665 INFO:teuthology.orchestra.run.smithi007.stdout:141733920822 2024-03-23T13:18:24.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-23T13:18:24.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.101+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/2612958652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f18100da0 msgr2=0x7f6f18101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.101+0000 7f6f1fcb6700 1 --2- 172.21.15.7:0/2612958652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f18100da0 0x7f6f18101180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6f100099c0 tx=0x7f6f10009cd0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.101+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/2612958652 shutdown_connections 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.101+0000 7f6f1fcb6700 1 --2- 172.21.15.7:0/2612958652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f181075c0 0x7f6f1810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.101+0000 7f6f1fcb6700 1 --2- 172.21.15.7:0/2612958652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f18100da0 0x7f6f18101180 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.101+0000 7f6f1fcb6700 1 --2- 172.21.15.7:0/2612958652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f18101750 0x7f6f18106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.101+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/2612958652 >> 172.21.15.7:0/2612958652 conn(0x7f6f18076440 msgr2=0x7f6f18076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.101+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/2612958652 shutdown_connections 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.101+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/2612958652 wait complete. 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1fcb6700 1 Processor -- start 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1fcb6700 1 -- start start 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1fcb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f18100da0 0x7f6f18101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1fcb6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f18101750 0x7f6f18102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:24.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1fcb6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f181075c0 0x7f6f181027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1fcb6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181a6bb0 con 0x7f6f181075c0 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1fcb6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181a6d20 con 0x7f6f18101750 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1fcb6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181a6ee0 con 0x7f6f18100da0 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1e253700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f181075c0 0x7f6f181027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1da52700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f18100da0 0x7f6f18101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1e253700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f181075c0 0x7f6f181027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42326/0 (socket says 172.21.15.7:42326) 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1e253700 1 -- 172.21.15.7:0/3324917146 learned_addr learned my addr 172.21.15.7:0/3324917146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1da52700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f18100da0 0x7f6f18101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:34514/0 (socket says 172.21.15.7:34514) 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1d251700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f18101750 0x7f6f18102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1e253700 1 -- 172.21.15.7:0/3324917146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f18100da0 msgr2=0x7f6f18101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1e253700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f18100da0 0x7f6f18101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1e253700 1 -- 172.21.15.7:0/3324917146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f18101750 msgr2=0x7f6f18102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1e253700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f18101750 0x7f6f18102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.102+0000 7f6f1e253700 1 -- 172.21.15.7:0/3324917146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f100095f0 con 0x7f6f181075c0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.103+0000 7f6f1e253700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f181075c0 0x7f6f181027a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800ef30 tx=0x7f6f0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.103+0000 7f6f16ffd700 1 -- 172.21.15.7:0/3324917146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08018070 con 0x7f6f181075c0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.103+0000 7f6f16ffd700 1 -- 172.21.15.7:0/3324917146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0800f040 con 0x7f6f181075c0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.103+0000 7f6f16ffd700 1 -- 172.21.15.7:0/3324917146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0800cd50 con 0x7f6f181075c0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.103+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f181a7160 con 0x7f6f181075c0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.103+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f181a7660 con 0x7f6f181075c0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.103+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f181125f0 con 0x7f6f181075c0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.105+0000 7f6f16ffd700 1 -- 172.21.15.7:0/3324917146 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f6f08007500 con 0x7f6f181075c0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.105+0000 7f6f16ffd700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6f000a21e0 0x7f6f000a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.105+0000 7f6f1da52700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6f000a21e0 0x7f6f000a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.106+0000 7f6f16ffd700 1 -- 172.21.15.7:0/3324917146 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f6f080ce0d0 con 0x7f6f181075c0 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.106+0000 7f6f16ffd700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] conn(0x7f6f000ab9c0 0x7f6f000adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:24.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.106+0000 7f6f16ffd700 1 -- 172.21.15.7:0/3324917146 --> [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f000ae490 con 0x7f6f000ab9c0 2024-03-23T13:18:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.106+0000 7f6f16ffd700 1 -- 172.21.15.7:0/3324917146 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f6f080ce570 con 0x7f6f181075c0 2024-03-23T13:18:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.106+0000 7f6f1da52700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6f000a21e0 0x7f6f000a46a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6f10000c00 tx=0x7f6f10005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.106+0000 7f6f1d251700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] conn(0x7f6f000ab9c0 0x7f6f000adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.106+0000 7f6f1d251700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] conn(0x7f6f000ab9c0 0x7f6f000adde0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.107+0000 7f6f16ffd700 1 -- 172.21.15.7:0/3324917146 <== osd.11 v2:172.21.15.155:6824/2887615966 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f000ae490 con 0x7f6f000ab9c0 2024-03-23T13:18:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.123+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 --> [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f18062910 con 0x7f6f000ab9c0 2024-03-23T13:18:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f16ffd700 1 -- 172.21.15.7:0/3324917146 <== osd.11 v2:172.21.15.155:6824/2887615966 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6f18062910 con 0x7f6f000ab9c0 2024-03-23T13:18:24.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 >> [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] conn(0x7f6f000ab9c0 msgr2=0x7f6f000adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1fcb6700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] conn(0x7f6f000ab9c0 0x7f6f000adde0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6f000a21e0 msgr2=0x7f6f000a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1fcb6700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f6f000a21e0 0x7f6f000a46a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6f10000c00 tx=0x7f6f10005fb0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f181075c0 msgr2=0x7f6f181027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1fcb6700 1 --2- 172.21.15.7:0/3324917146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f181075c0 0x7f6f181027a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800ef30 tx=0x7f6f0800c3a0 comp rx=0 tx=0).stop 2024-03-23T13:18:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1e253700 1 -- 172.21.15.7:0/3324917146 reap_dead start 2024-03-23T13:18:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 shutdown_connections 2024-03-23T13:18:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 >> 172.21.15.7:0/3324917146 conn(0x7f6f18076440 msgr2=0x7f6f18100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 shutdown_connections 2024-03-23T13:18:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:21.124+0000 7f6f1fcb6700 1 -- 172.21.15.7:0/3324917146 wait complete. 2024-03-23T13:18:25.455 INFO:teuthology.orchestra.run.smithi007.stdout:343597383700 2024-03-23T13:18:25.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-23T13:18:25.758 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:25 smithi007 ceph-mon[32184]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T13:18:25.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:25 smithi130 ceph-mon[36225]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T13:18:26.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:25 smithi155 ceph-mon[36157]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T13:18:26.044 INFO:teuthology.orchestra.run.smithi007.stdout:287762808863 2024-03-23T13:18:26.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-23T13:18:27.130 INFO:teuthology.orchestra.run.smithi007.stdout:253403070501 2024-03-23T13:18:27.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-23T13:18:27.678 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:27 smithi007 ceph-mon[32184]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-23T13:18:27.680 INFO:teuthology.orchestra.run.smithi007.stdout:197568495662 2024-03-23T13:18:27.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-23T13:18:27.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:27 smithi130 ceph-mon[36225]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-23T13:18:28.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:27 smithi155 ceph-mon[36157]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-23T13:18:28.143 INFO:teuthology.orchestra.run.smithi007.stdout:309237645336 2024-03-23T13:18:28.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-23T13:18:28.600 INFO:teuthology.orchestra.run.smithi007.stdout:219043332137 2024-03-23T13:18:28.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-23T13:18:28.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.855+0000 7f3b77d79700 1 -- 172.21.15.7:0/1010640678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70107520 msgr2=0x7f3b7010f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:28.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.855+0000 7f3b77d79700 1 --2- 172.21.15.7:0/1010640678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70107520 0x7f3b7010f0f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800b0a0 tx=0x7f3b6800b3b0 comp rx=0 tx=0).stop 2024-03-23T13:18:28.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.855+0000 7f3b77d79700 1 -- 172.21.15.7:0/1010640678 shutdown_connections 2024-03-23T13:18:28.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.855+0000 7f3b77d79700 1 --2- 172.21.15.7:0/1010640678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70107520 0x7f3b7010f0f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:28.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.855+0000 7f3b77d79700 1 --2- 172.21.15.7:0/1010640678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b70102440 0x7f3b70102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.855+0000 7f3b77d79700 1 --2- 172.21.15.7:0/1010640678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b70102df0 0x7f3b70106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.855+0000 7f3b77d79700 1 -- 172.21.15.7:0/1010640678 >> 172.21.15.7:0/1010640678 conn(0x7f3b700fdcb0 msgr2=0x7f3b701000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.855+0000 7f3b77d79700 1 -- 172.21.15.7:0/1010640678 shutdown_connections 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.855+0000 7f3b77d79700 1 -- 172.21.15.7:0/1010640678 wait complete. 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b77d79700 1 Processor -- start 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b77d79700 1 -- start start 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b77d79700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70102440 0x7f3b70074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b77d79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b70102df0 0x7f3b70074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b77d79700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b70107520 0x7f3b70074f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b77d79700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701a6c50 con 0x7f3b70102440 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b77d79700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701a6dc0 con 0x7f3b70107520 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b77d79700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701a6f80 con 0x7f3b70102df0 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b75b15700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70102440 0x7f3b70074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b75b15700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70102440 0x7f3b70074500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48462/0 (socket says 172.21.15.7:48462) 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b75b15700 1 -- 172.21.15.7:0/1218790624 learned_addr learned my addr 172.21.15.7:0/1218790624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b76316700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b70107520 0x7f3b70074f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b75314700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b70102df0 0x7f3b70074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b75b15700 1 -- 172.21.15.7:0/1218790624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b70102df0 msgr2=0x7f3b70074a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b75b15700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b70102df0 0x7f3b70074a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b75b15700 1 -- 172.21.15.7:0/1218790624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b70107520 msgr2=0x7f3b70074f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b75b15700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b70107520 0x7f3b70074f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b75b15700 1 -- 172.21.15.7:0/1218790624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b68009d00 con 0x7f3b70102440 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.856+0000 7f3b76316700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3b70107520 0x7f3b70074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.857+0000 7f3b75b15700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70102440 0x7f3b70074500 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00e9a0 tx=0x7f3b6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.857+0000 7f3b62ffd700 1 -- 172.21.15.7:0/1218790624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c009790 con 0x7f3b70102440 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.857+0000 7f3b62ffd700 1 -- 172.21.15.7:0/1218790624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b6c010430 con 0x7f3b70102440 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.857+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b701a7260 con 0x7f3b70102440 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.857+0000 7f3b62ffd700 1 -- 172.21.15.7:0/1218790624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c018680 con 0x7f3b70102440 2024-03-23T13:18:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.857+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b701a7760 con 0x7f3b70102440 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.858+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b70112550 con 0x7f3b70102440 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.859+0000 7f3b62ffd700 1 -- 172.21.15.7:0/1218790624 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f3b6c010910 con 0x7f3b70102440 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.859+0000 7f3b62ffd700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3b5c0a21e0 0x7f3b5c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.860+0000 7f3b75314700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3b5c0a21e0 0x7f3b5c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.860+0000 7f3b62ffd700 1 -- 172.21.15.7:0/1218790624 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3b6c014070 con 0x7f3b70102440 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.860+0000 7f3b62ffd700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] conn(0x7f3b5c0ab9c0 0x7f3b5c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.860+0000 7f3b62ffd700 1 -- 172.21.15.7:0/1218790624 --> [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b5c0ae490 con 0x7f3b5c0ab9c0 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.860+0000 7f3b62ffd700 1 -- 172.21.15.7:0/1218790624 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f3b6c0ce910 con 0x7f3b70102440 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.860+0000 7f3b76316700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] conn(0x7f3b5c0ab9c0 0x7f3b5c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.860+0000 7f3b75314700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3b5c0a21e0 0x7f3b5c0a46a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3b70108a50 tx=0x7f3b64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.860+0000 7f3b76316700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] conn(0x7f3b5c0ab9c0 0x7f3b5c0adde0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.861+0000 7f3b62ffd700 1 -- 172.21.15.7:0/1218790624 <== osd.0 v2:172.21.15.7:6802/2113068565 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b5c0ae490 con 0x7f3b5c0ab9c0 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.879+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 --> [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b70062910 con 0x7f3b5c0ab9c0 2024-03-23T13:18:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.879+0000 7f3b62ffd700 1 -- 172.21.15.7:0/1218790624 <== osd.0 v2:172.21.15.7:6802/2113068565 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3b70062910 con 0x7f3b5c0ab9c0 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] conn(0x7f3b5c0ab9c0 msgr2=0x7f3b5c0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b77d79700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] conn(0x7f3b5c0ab9c0 0x7f3b5c0adde0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3b5c0a21e0 msgr2=0x7f3b5c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b77d79700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f3b5c0a21e0 0x7f3b5c0a46a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3b70108a50 tx=0x7f3b64009260 comp rx=0 tx=0).stop 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70102440 msgr2=0x7f3b70074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b77d79700 1 --2- 172.21.15.7:0/1218790624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b70102440 0x7f3b70074500 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00e9a0 tx=0x7f3b6c00ecb0 comp rx=0 tx=0).stop 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b76316700 1 -- 172.21.15.7:0/1218790624 reap_dead start 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 shutdown_connections 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 >> 172.21.15.7:0/1218790624 conn(0x7f3b700fdcb0 msgr2=0x7f3b700fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 shutdown_connections 2024-03-23T13:18:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:24.880+0000 7f3b77d79700 1 -- 172.21.15.7:0/1218790624 wait complete. 2024-03-23T13:18:29.309 INFO:teuthology.orchestra.run.smithi007.stdout:34359738445 2024-03-23T13:18:29.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-23T13:18:29.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:29 smithi007 ceph-mon[32184]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:18:29.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.843+0000 7f9086be7700 1 -- 172.21.15.7:0/3205032764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080107520 msgr2=0x7f908010f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:29.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.843+0000 7f9086be7700 1 --2- 172.21.15.7:0/3205032764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080107520 0x7f908010f0f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f907800b0a0 tx=0x7f907800b3b0 comp rx=0 tx=0).stop 2024-03-23T13:18:29.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.843+0000 7f9086be7700 1 -- 172.21.15.7:0/3205032764 shutdown_connections 2024-03-23T13:18:29.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.843+0000 7f9086be7700 1 --2- 172.21.15.7:0/3205032764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080107520 0x7f908010f0f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:29.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.843+0000 7f9086be7700 1 --2- 172.21.15.7:0/3205032764 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9080102440 0x7f9080102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:29.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.843+0000 7f9086be7700 1 --2- 172.21.15.7:0/3205032764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9080102df0 0x7f9080106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:29.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.843+0000 7f9086be7700 1 -- 172.21.15.7:0/3205032764 >> 172.21.15.7:0/3205032764 conn(0x7f90800fdcb0 msgr2=0x7f90801000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:29.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.843+0000 7f9086be7700 1 -- 172.21.15.7:0/3205032764 shutdown_connections 2024-03-23T13:18:29.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.843+0000 7f9086be7700 1 -- 172.21.15.7:0/3205032764 wait complete. 2024-03-23T13:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9086be7700 1 Processor -- start 2024-03-23T13:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9086be7700 1 -- start start 2024-03-23T13:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9086be7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080102440 0x7f9080074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9086be7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9080102df0 0x7f9080074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9086be7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9080107520 0x7f9080074f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9086be7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90801a6c10 con 0x7f9080102440 2024-03-23T13:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9086be7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f90801a6d80 con 0x7f9080107520 2024-03-23T13:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9086be7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f90801a6f40 con 0x7f9080102df0 2024-03-23T13:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9084983700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080102440 0x7f9080074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9084983700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080102440 0x7f9080074500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48488/0 (socket says 172.21.15.7:48488) 2024-03-23T13:18:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9084983700 1 -- 172.21.15.7:0/4243855792 learned_addr learned my addr 172.21.15.7:0/4243855792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f907ffff700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9080102df0 0x7f9080074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.844+0000 7f9085184700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9080107520 0x7f9080074f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f9084983700 1 -- 172.21.15.7:0/4243855792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9080102df0 msgr2=0x7f9080074a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f9084983700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9080102df0 0x7f9080074a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f9084983700 1 -- 172.21.15.7:0/4243855792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9080107520 msgr2=0x7f9080074f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f9084983700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9080107520 0x7f9080074f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f9084983700 1 -- 172.21.15.7:0/4243855792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9078009d00 con 0x7f9080102440 2024-03-23T13:18:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f9084983700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080102440 0x7f9080074500 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f907400e9a0 tx=0x7f907400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f907dffb700 1 -- 172.21.15.7:0/4243855792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074009790 con 0x7f9080102440 2024-03-23T13:18:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f907dffb700 1 -- 172.21.15.7:0/4243855792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9074010430 con 0x7f9080102440 2024-03-23T13:18:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90801a7220 con 0x7f9080102440 2024-03-23T13:18:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f907dffb700 1 -- 172.21.15.7:0/4243855792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074018680 con 0x7f9080102440 2024-03-23T13:18:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.845+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90801a7770 con 0x7f9080102440 2024-03-23T13:18:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.846+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9080112550 con 0x7f9080102440 2024-03-23T13:18:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.847+0000 7f907dffb700 1 -- 172.21.15.7:0/4243855792 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f9074010910 con 0x7f9080102440 2024-03-23T13:18:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.848+0000 7f907dffb700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f90700a2260 0x7f90700a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.848+0000 7f907ffff700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f90700a2260 0x7f90700a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.848+0000 7f907dffb700 1 -- 172.21.15.7:0/4243855792 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9074014070 con 0x7f9080102440 2024-03-23T13:18:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.848+0000 7f907dffb700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] conn(0x7f90700aba40 0x7f90700ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.848+0000 7f907dffb700 1 -- 172.21.15.7:0/4243855792 --> [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90700ae510 con 0x7f90700aba40 2024-03-23T13:18:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.848+0000 7f907dffb700 1 -- 172.21.15.7:0/4243855792 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f90740cecb0 con 0x7f9080102440 2024-03-23T13:18:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.848+0000 7f907ffff700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f90700a2260 0x7f90700a4720 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9080101410 tx=0x7f906c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.848+0000 7f9085184700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] conn(0x7f90700aba40 0x7f90700ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.849+0000 7f9085184700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] conn(0x7f90700aba40 0x7f90700ade60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.849+0000 7f907dffb700 1 -- 172.21.15.7:0/4243855792 <== osd.2 v2:172.21.15.155:6800/2256065708 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90700ae510 con 0x7f90700aba40 2024-03-23T13:18:29.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.868+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 --> [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9080062890 con 0x7f90700aba40 2024-03-23T13:18:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.868+0000 7f907dffb700 1 -- 172.21.15.7:0/4243855792 <== osd.2 v2:172.21.15.155:6800/2256065708 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9080062890 con 0x7f90700aba40 2024-03-23T13:18:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.868+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 >> [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] conn(0x7f90700aba40 msgr2=0x7f90700ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.868+0000 7f9086be7700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] conn(0x7f90700aba40 0x7f90700ade60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.869+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f90700a2260 msgr2=0x7f90700a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.869+0000 7f9086be7700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f90700a2260 0x7f90700a4720 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9080101410 tx=0x7f906c009260 comp rx=0 tx=0).stop 2024-03-23T13:18:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.869+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080102440 msgr2=0x7f9080074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.869+0000 7f9086be7700 1 --2- 172.21.15.7:0/4243855792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080102440 0x7f9080074500 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f907400e9a0 tx=0x7f907400ecb0 comp rx=0 tx=0).stop 2024-03-23T13:18:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.869+0000 7f9085184700 1 -- 172.21.15.7:0/4243855792 reap_dead start 2024-03-23T13:18:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.869+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 shutdown_connections 2024-03-23T13:18:29.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.869+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 >> 172.21.15.7:0/4243855792 conn(0x7f90800fdcb0 msgr2=0x7f90800fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:29.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.869+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 shutdown_connections 2024-03-23T13:18:29.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:29.869+0000 7f9086be7700 1 -- 172.21.15.7:0/4243855792 wait complete. 2024-03-23T13:18:29.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:29 smithi130 ceph-mon[36225]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:18:30.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:29 smithi155 ceph-mon[36157]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:18:30.126 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:30.127 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:30.134 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:30.663 INFO:teuthology.orchestra.run.smithi007.stdout:77309411396 2024-03-23T13:18:30.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-23T13:18:31.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:31 smithi130 ceph-mon[36225]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:31.953 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:31 smithi007 ceph-mon[32184]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:32.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:31 smithi155 ceph-mon[36157]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:33.746 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:33.746 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:33.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:33 smithi130 ceph-mon[36225]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:18:34.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:33 smithi155 ceph-mon[36157]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:18:34.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:33 smithi007 ceph-mon[32184]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:18:35.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:35 smithi130 ceph-mon[36225]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-23T13:18:35.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:35 smithi007 ceph-mon[32184]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-23T13:18:36.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:35 smithi155 ceph-mon[36157]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-23T13:18:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.346+0000 7f9ad7611700 1 -- 172.21.15.7:0/1646877267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad006a850 msgr2=0x7f9ad010f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.346+0000 7f9ad7611700 1 --2- 172.21.15.7:0/1646877267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad006a850 0x7f9ad010f090 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80099c0 tx=0x7f9ac8009cd0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.346+0000 7f9ad7611700 1 -- 172.21.15.7:0/1646877267 shutdown_connections 2024-03-23T13:18:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.346+0000 7f9ad7611700 1 --2- 172.21.15.7:0/1646877267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad006a850 0x7f9ad010f090 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.346+0000 7f9ad7611700 1 --2- 172.21.15.7:0/1646877267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ad0108250 0x7f9ad0108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.346+0000 7f9ad7611700 1 --2- 172.21.15.7:0/1646877267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ad0069eb0 0x7f9ad006a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.346+0000 7f9ad7611700 1 -- 172.21.15.7:0/1646877267 >> 172.21.15.7:0/1646877267 conn(0x7f9ad0078030 msgr2=0x7f9ad0078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.346+0000 7f9ad7611700 1 -- 172.21.15.7:0/1646877267 shutdown_connections 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.346+0000 7f9ad7611700 1 -- 172.21.15.7:0/1646877267 wait complete. 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad7611700 1 Processor -- start 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad7611700 1 -- start start 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad7611700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad0069eb0 0x7f9ad01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad7611700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ad006a850 0x7f9ad0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad7611700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ad0108250 0x7f9ad019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad7611700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad019c5a0 con 0x7f9ad0069eb0 2024-03-23T13:18:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad7611700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad019c710 con 0x7f9ad006a850 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad7611700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad019c8d0 con 0x7f9ad0108250 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad53ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad0069eb0 0x7f9ad01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad53ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad0069eb0 0x7f9ad01987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49946/0 (socket says 172.21.15.7:49946) 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad53ad700 1 -- 172.21.15.7:0/936303126 learned_addr learned my addr 172.21.15.7:0/936303126 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad5bae700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ad0108250 0x7f9ad019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.347+0000 7f9ad4bac700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ad006a850 0x7f9ad0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ad53ad700 1 -- 172.21.15.7:0/936303126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ad0108250 msgr2=0x7f9ad019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ad53ad700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ad0108250 0x7f9ad019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ad53ad700 1 -- 172.21.15.7:0/936303126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ad006a850 msgr2=0x7f9ad0198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ad53ad700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ad006a850 0x7f9ad0198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ad53ad700 1 -- 172.21.15.7:0/936303126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac80095f0 con 0x7f9ad0069eb0 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ad53ad700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad0069eb0 0x7f9ad01987d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000ef10 tx=0x7f9ac000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ace7fc700 1 -- 172.21.15.7:0/936303126 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000c9c0 con 0x7f9ad0069eb0 2024-03-23T13:18:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ad7611700 1 -- 172.21.15.7:0/936303126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad019cb50 con 0x7f9ad0069eb0 2024-03-23T13:18:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ace7fc700 1 -- 172.21.15.7:0/936303126 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac000f040 con 0x7f9ad0069eb0 2024-03-23T13:18:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ace7fc700 1 -- 172.21.15.7:0/936303126 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0014480 con 0x7f9ad0069eb0 2024-03-23T13:18:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.348+0000 7f9ad7611700 1 -- 172.21.15.7:0/936303126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad01a5500 con 0x7f9ad0069eb0 2024-03-23T13:18:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.349+0000 7f9ad7611700 1 -- 172.21.15.7:0/936303126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad010a770 con 0x7f9ad0069eb0 2024-03-23T13:18:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.350+0000 7f9ace7fc700 1 -- 172.21.15.7:0/936303126 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f9ac0007720 con 0x7f9ad0069eb0 2024-03-23T13:18:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.351+0000 7f9ace7fc700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9ab80a21e0 0x7f9ab80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.351+0000 7f9ad4bac700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9ab80a21e0 0x7f9ab80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.351+0000 7f9ace7fc700 1 -- 172.21.15.7:0/936303126 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9ac00cdfc0 con 0x7f9ad0069eb0 2024-03-23T13:18:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.351+0000 7f9ad4bac700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9ab80a21e0 0x7f9ab80a46a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0078530 tx=0x7f9ac40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.355+0000 7f9ace7fc700 1 -- 172.21.15.7:0/936303126 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9ac0096ee0 con 0x7f9ad0069eb0 2024-03-23T13:18:36.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.559+0000 7f9ad7611700 1 -- 172.21.15.7:0/936303126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9ad0199ae0 con 0x7f9ad0069eb0 2024-03-23T13:18:36.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.559+0000 7f9ace7fc700 1 -- 172.21.15.7:0/936303126 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9ac009a5a0 con 0x7f9ad0069eb0 2024-03-23T13:18:36.560 INFO:teuthology.orchestra.run.smithi007.stdout:55834574922 2024-03-23T13:18:36.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 -- 172.21.15.7:0/936303126 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9ab80a21e0 msgr2=0x7f9ab80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:36.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9ab80a21e0 0x7f9ab80a46a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0078530 tx=0x7f9ac40073d0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 -- 172.21.15.7:0/936303126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad0069eb0 msgr2=0x7f9ad01987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:36.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad0069eb0 0x7f9ad01987d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000ef10 tx=0x7f9ac000c3a0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 -- 172.21.15.7:0/936303126 shutdown_connections 2024-03-23T13:18:36.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9ab80a21e0 0x7f9ab80a46a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ad0108250 0x7f9ad019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad0069eb0 0x7f9ad01987d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 --2- 172.21.15.7:0/936303126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ad006a850 0x7f9ad0198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:36.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 -- 172.21.15.7:0/936303126 >> 172.21.15.7:0/936303126 conn(0x7f9ad0078030 msgr2=0x7f9ad00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:36.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 -- 172.21.15.7:0/936303126 shutdown_connections 2024-03-23T13:18:36.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:36.561+0000 7f9ad7611700 1 -- 172.21.15.7:0/936303126 wait complete. 2024-03-23T13:18:36.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:36 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/936303126' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T13:18:36.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:36 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/936303126' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T13:18:37.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:36 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/936303126' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T13:18:37.166 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:37.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:37 smithi007 ceph-mon[32184]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:37.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:37 smithi130 ceph-mon[36225]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:38.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:37 smithi155 ceph-mon[36157]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.807+0000 7f9a915b8700 1 -- 172.21.15.7:0/1354668755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c1024b0 msgr2=0x7f9a8c102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.807+0000 7f9a915b8700 1 --2- 172.21.15.7:0/1354668755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c1024b0 0x7f9a8c102890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009990 tx=0x7f9a7c009ca0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.807+0000 7f9a915b8700 1 -- 172.21.15.7:0/1354668755 shutdown_connections 2024-03-23T13:18:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.807+0000 7f9a915b8700 1 --2- 172.21.15.7:0/1354668755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a8c107500 0x7f9a8c10f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.807+0000 7f9a915b8700 1 --2- 172.21.15.7:0/1354668755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c1024b0 0x7f9a8c102890 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.807+0000 7f9a915b8700 1 --2- 172.21.15.7:0/1354668755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c102e60 0x7f9a8c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.807+0000 7f9a915b8700 1 -- 172.21.15.7:0/1354668755 >> 172.21.15.7:0/1354668755 conn(0x7f9a8c079360 msgr2=0x7f9a8c079770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.807+0000 7f9a915b8700 1 -- 172.21.15.7:0/1354668755 shutdown_connections 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.807+0000 7f9a915b8700 1 -- 172.21.15.7:0/1354668755 wait complete. 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.808+0000 7f9a915b8700 1 Processor -- start 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.808+0000 7f9a915b8700 1 -- start start 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.808+0000 7f9a915b8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c1024b0 0x7f9a8c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.808+0000 7f9a915b8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c102e60 0x7f9a8c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.808+0000 7f9a915b8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a8c107500 0x7f9a8c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.808+0000 7f9a915b8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c19d600 con 0x7f9a8c102e60 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.808+0000 7f9a915b8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c19d770 con 0x7f9a8c107500 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.808+0000 7f9a915b8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c19d930 con 0x7f9a8c1024b0 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.808+0000 7f9a8a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c102e60 0x7f9a8c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c1024b0 0x7f9a8c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c102e60 0x7f9a8c198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49962/0 (socket says 172.21.15.7:49962) 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8b7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a8c107500 0x7f9a8c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8a7fc700 1 -- 172.21.15.7:0/3817497899 learned_addr learned my addr 172.21.15.7:0/3817497899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8b7fe700 1 -- 172.21.15.7:0/3817497899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c1024b0 msgr2=0x7f9a8c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8b7fe700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c1024b0 0x7f9a8c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8b7fe700 1 -- 172.21.15.7:0/3817497899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c102e60 msgr2=0x7f9a8c198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8b7fe700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c102e60 0x7f9a8c198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8b7fe700 1 -- 172.21.15.7:0/3817497899 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c0095f0 con 0x7f9a8c107500 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8affd700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c1024b0 0x7f9a8c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8a7fc700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c102e60 0x7f9a8c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a8b7fe700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a8c107500 0x7f9a8c19cff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400efc0 tx=0x7f9a8400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:39.153 INFO:teuthology.orchestra.run.smithi007.stdout:219043332140 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a73fff700 1 -- 172.21.15.7:0/3817497899 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84018070 con 0x7f9a8c107500 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a915b8700 1 -- 172.21.15.7:0/3817497899 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c19dbb0 con 0x7f9a8c107500 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a73fff700 1 -- 172.21.15.7:0/3817497899 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8400f040 con 0x7f9a8c107500 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a73fff700 1 -- 172.21.15.7:0/3817497899 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8400cd10 con 0x7f9a8c107500 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.809+0000 7f9a915b8700 1 -- 172.21.15.7:0/3817497899 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c1a5470 con 0x7f9a8c107500 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.810+0000 7f9a915b8700 1 -- 172.21.15.7:0/3817497899 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c10a7c0 con 0x7f9a8c107500 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.811+0000 7f9a73fff700 1 -- 172.21.15.7:0/3817497899 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f9a84004ad0 con 0x7f9a8c107500 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.812+0000 7f9a73fff700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a740a21e0 0x7f9a740a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.812+0000 7f9a8affd700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a740a21e0 0x7f9a740a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.812+0000 7f9a73fff700 1 -- 172.21.15.7:0/3817497899 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9a840cd9a0 con 0x7f9a8c107500 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.812+0000 7f9a8affd700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a740a21e0 0x7f9a740a46a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00b2e0 tx=0x7f9a7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:38.816+0000 7f9a73fff700 1 -- 172.21.15.7:0/3817497899 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9a84096950 con 0x7f9a8c107500 2024-03-23T13:18:39.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.024+0000 7f9a915b8700 1 -- 172.21.15.7:0/3817497899 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9a8c199a80 con 0x7f9a8c107500 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.025+0000 7f9a73fff700 1 -- 172.21.15.7:0/3817497899 <== mon.2 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9a8409a010 con 0x7f9a8c107500 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 -- 172.21.15.7:0/3817497899 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a740a21e0 msgr2=0x7f9a740a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a740a21e0 0x7f9a740a46a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00b2e0 tx=0x7f9a7c005fd0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 -- 172.21.15.7:0/3817497899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a8c107500 msgr2=0x7f9a8c19cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a8c107500 0x7f9a8c19cff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400efc0 tx=0x7f9a8400c3a0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 -- 172.21.15.7:0/3817497899 shutdown_connections 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f9a740a21e0 0x7f9a740a46a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a8c107500 0x7f9a8c19cff0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c1024b0 0x7f9a8c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 --2- 172.21.15.7:0/3817497899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c102e60 0x7f9a8c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 -- 172.21.15.7:0/3817497899 >> 172.21.15.7:0/3817497899 conn(0x7f9a8c079360 msgr2=0x7f9a8c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 -- 172.21.15.7:0/3817497899 shutdown_connections 2024-03-23T13:18:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:39.027+0000 7f9a915b8700 1 -- 172.21.15.7:0/3817497899 wait complete. 2024-03-23T13:18:39.800 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:39.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:39 smithi130 ceph-mon[36225]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:18:39.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:39 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3817497899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T13:18:39.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:39 smithi007 ceph-mon[32184]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:18:39.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:39 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3817497899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T13:18:40.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:39 smithi155 ceph-mon[36157]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:18:40.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:39 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3817497899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T13:18:40.560 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:40.560 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:40.992 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574922 for osd.1 2024-03-23T13:18:40.993 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:41.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.413+0000 7fa51d13d700 1 -- 172.21.15.7:0/1365787361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5181022b0 msgr2=0x7fa518102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:41.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.413+0000 7fa51d13d700 1 --2- 172.21.15.7:0/1365787361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5181022b0 0x7fa518102730 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa500009990 tx=0x7fa500009ca0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.413+0000 7fa51d13d700 1 -- 172.21.15.7:0/1365787361 shutdown_connections 2024-03-23T13:18:41.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.413+0000 7fa51d13d700 1 --2- 172.21.15.7:0/1365787361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa518102c70 0x7fa51810f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.413+0000 7fa51d13d700 1 --2- 172.21.15.7:0/1365787361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa518108320 0x7fa518108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.413+0000 7fa51d13d700 1 --2- 172.21.15.7:0/1365787361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5181022b0 0x7fa518102730 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.413+0000 7fa51d13d700 1 -- 172.21.15.7:0/1365787361 >> 172.21.15.7:0/1365787361 conn(0x7fa5180792d0 msgr2=0x7fa5180796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.413+0000 7fa51d13d700 1 -- 172.21.15.7:0/1365787361 shutdown_connections 2024-03-23T13:18:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.413+0000 7fa51d13d700 1 -- 172.21.15.7:0/1365787361 wait complete. 2024-03-23T13:18:41.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.414+0000 7fa51d13d700 1 Processor -- start 2024-03-23T13:18:41.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.414+0000 7fa51d13d700 1 -- start start 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.414+0000 7fa51d13d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5181022b0 0x7fa51819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.414+0000 7fa51d13d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa518102c70 0x7fa51819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.414+0000 7fa51d13d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa518108320 0x7fa518198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.414+0000 7fa51d13d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa518198c60 con 0x7fa5181022b0 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.414+0000 7fa51d13d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa518198da0 con 0x7fa518108320 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.414+0000 7fa51d13d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa518198f60 con 0x7fa518102c70 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.414+0000 7fa516d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5181022b0 0x7fa51819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa516d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5181022b0 0x7fa51819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49974/0 (socket says 172.21.15.7:49974) 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa516d9d700 1 -- 172.21.15.7:0/1318824170 learned_addr learned my addr 172.21.15.7:0/1318824170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa51659c700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa518102c70 0x7fa51819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa51759e700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa518108320 0x7fa518198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa516d9d700 1 -- 172.21.15.7:0/1318824170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa518102c70 msgr2=0x7fa51819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa516d9d700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa518102c70 0x7fa51819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa516d9d700 1 -- 172.21.15.7:0/1318824170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa518108320 msgr2=0x7fa518198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:41.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa516d9d700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa518108320 0x7fa518198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa516d9d700 1 -- 172.21.15.7:0/1318824170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5000095f0 con 0x7fa5181022b0 2024-03-23T13:18:41.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa51659c700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa518102c70 0x7fa51819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:41.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa51759e700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa518108320 0x7fa518198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:41.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa516d9d700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5181022b0 0x7fa51819e3f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa50800ef10 tx=0x7fa50800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:41.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa50ffff700 1 -- 172.21.15.7:0/1318824170 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50800c9c0 con 0x7fa5181022b0 2024-03-23T13:18:41.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa51d13d700 1 -- 172.21.15.7:0/1318824170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5181991e0 con 0x7fa5181022b0 2024-03-23T13:18:41.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa50ffff700 1 -- 172.21.15.7:0/1318824170 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa50800f040 con 0x7fa5181022b0 2024-03-23T13:18:41.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.415+0000 7fa50ffff700 1 -- 172.21.15.7:0/1318824170 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508014480 con 0x7fa5181022b0 2024-03-23T13:18:41.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.416+0000 7fa51d13d700 1 -- 172.21.15.7:0/1318824170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5181a5490 con 0x7fa5181022b0 2024-03-23T13:18:41.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.417+0000 7fa51d13d700 1 -- 172.21.15.7:0/1318824170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51810a740 con 0x7fa5181022b0 2024-03-23T13:18:41.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.418+0000 7fa50ffff700 1 -- 172.21.15.7:0/1318824170 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fa508007720 con 0x7fa5181022b0 2024-03-23T13:18:41.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.418+0000 7fa50ffff700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa5040a21e0 0x7fa5040a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:41.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.418+0000 7fa51659c700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa5040a21e0 0x7fa5040a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.418+0000 7fa50ffff700 1 -- 172.21.15.7:0/1318824170 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa5080cdea0 con 0x7fa5181022b0 2024-03-23T13:18:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.419+0000 7fa51659c700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa5040a21e0 0x7fa5040a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa50000b2e0 tx=0x7fa500005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.423+0000 7fa50ffff700 1 -- 172.21.15.7:0/1318824170 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa508096ec0 con 0x7fa5181022b0 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.594+0000 7f84dd463700 1 -- 172.21.15.7:0/3515883368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8101aa0 msgr2=0x7f84d810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.594+0000 7f84dd463700 1 --2- 172.21.15.7:0/3515883368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8101aa0 0x7f84d810df40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f84d000b0a0 tx=0x7f84d000b3b0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.594+0000 7f84dd463700 1 -- 172.21.15.7:0/3515883368 shutdown_connections 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.594+0000 7f84dd463700 1 --2- 172.21.15.7:0/3515883368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8101aa0 0x7f84d810df40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.594+0000 7f84dd463700 1 --2- 172.21.15.7:0/3515883368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84d8107190 0x7f84d8107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.594+0000 7f84dd463700 1 --2- 172.21.15.7:0/3515883368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d81010e0 0x7f84d8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.594+0000 7f84dd463700 1 -- 172.21.15.7:0/3515883368 >> 172.21.15.7:0/3515883368 conn(0x7f84d8078110 msgr2=0x7f84d8078520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.594+0000 7f84dd463700 1 -- 172.21.15.7:0/3515883368 shutdown_connections 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.594+0000 7f84dd463700 1 -- 172.21.15.7:0/3515883368 wait complete. 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84dd463700 1 Processor -- start 2024-03-23T13:18:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84dd463700 1 -- start start 2024-03-23T13:18:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84dd463700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84d81010e0 0x7f84d8102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84dd463700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8101aa0 0x7f84d81032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84dd463700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d8107190 0x7f84d8103810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84dd463700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81a6b60 con 0x7f84d8101aa0 2024-03-23T13:18:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84dd463700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81a6cd0 con 0x7f84d81010e0 2024-03-23T13:18:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84dd463700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81a6e90 con 0x7f84d8107190 2024-03-23T13:18:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84d67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8101aa0 0x7f84d81032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84d67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8101aa0 0x7f84d81032d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49998/0 (socket says 172.21.15.7:49998) 2024-03-23T13:18:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84d67fc700 1 -- 172.21.15.7:0/2218256128 learned_addr learned my addr 172.21.15.7:0/2218256128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84d6ffd700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84d81010e0 0x7f84d8102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.595+0000 7f84d77fe700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d8107190 0x7f84d8103810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84d67fc700 1 -- 172.21.15.7:0/2218256128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d8107190 msgr2=0x7f84d8103810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84d67fc700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d8107190 0x7f84d8103810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84d67fc700 1 -- 172.21.15.7:0/2218256128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84d81010e0 msgr2=0x7f84d8102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84d67fc700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84d81010e0 0x7f84d8102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84d67fc700 1 -- 172.21.15.7:0/2218256128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d0009d00 con 0x7f84d8101aa0 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84d77fe700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d8107190 0x7f84d8103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84d6ffd700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84d81010e0 0x7f84d8102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84d67fc700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8101aa0 0x7f84d81032d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e9a0 tx=0x7f84cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84bffff700 1 -- 172.21.15.7:0/2218256128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc009790 con 0x7f84d8101aa0 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84dd463700 1 -- 172.21.15.7:0/2218256128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d81a7170 con 0x7f84d8101aa0 2024-03-23T13:18:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84bffff700 1 -- 172.21.15.7:0/2218256128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84cc010430 con 0x7f84d8101aa0 2024-03-23T13:18:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84dd463700 1 -- 172.21.15.7:0/2218256128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d81a76c0 con 0x7f84d8101aa0 2024-03-23T13:18:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.596+0000 7f84bffff700 1 -- 172.21.15.7:0/2218256128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc018680 con 0x7f84d8101aa0 2024-03-23T13:18:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.597+0000 7f84dd463700 1 -- 172.21.15.7:0/2218256128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d8068ec0 con 0x7f84d8101aa0 2024-03-23T13:18:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.598+0000 7f84bffff700 1 -- 172.21.15.7:0/2218256128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f84cc010910 con 0x7f84d8101aa0 2024-03-23T13:18:41.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.599+0000 7f84bffff700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84c00a2260 0x7f84c00a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:41.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.599+0000 7f84d6ffd700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84c00a2260 0x7f84c00a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:41.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.599+0000 7f84bffff700 1 -- 172.21.15.7:0/2218256128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f84cc014070 con 0x7f84d8101aa0 2024-03-23T13:18:41.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.599+0000 7f84d6ffd700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84c00a2260 0x7f84c00a4720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009fd0 tx=0x7f84c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:41.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.603+0000 7f84bffff700 1 -- 172.21.15.7:0/2218256128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f84cc053280 con 0x7f84d8101aa0 2024-03-23T13:18:41.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.632+0000 7fa51d13d700 1 -- 172.21.15.7:0/1318824170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fa518199ae0 con 0x7fa5181022b0 2024-03-23T13:18:41.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.632+0000 7fa50ffff700 1 -- 172.21.15.7:0/1318824170 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa50809a580 con 0x7fa5181022b0 2024-03-23T13:18:41.634 INFO:teuthology.orchestra.run.smithi007.stdout:287762808867 2024-03-23T13:18:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.634+0000 7fa51d13d700 1 -- 172.21.15.7:0/1318824170 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa5040a21e0 msgr2=0x7fa5040a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.634+0000 7fa51d13d700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa5040a21e0 0x7fa5040a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa50000b2e0 tx=0x7fa500005960 comp rx=0 tx=0).stop 2024-03-23T13:18:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.634+0000 7fa51d13d700 1 -- 172.21.15.7:0/1318824170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5181022b0 msgr2=0x7fa51819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.634+0000 7fa51d13d700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5181022b0 0x7fa51819e3f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa50800ef10 tx=0x7fa50800c3a0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.634+0000 7fa51d13d700 1 -- 172.21.15.7:0/1318824170 shutdown_connections 2024-03-23T13:18:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.634+0000 7fa51d13d700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa5040a21e0 0x7fa5040a46a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.634+0000 7fa51d13d700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa518108320 0x7fa518198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.634+0000 7fa51d13d700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5181022b0 0x7fa51819e3f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.634+0000 7fa51d13d700 1 --2- 172.21.15.7:0/1318824170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa518102c70 0x7fa51819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.634+0000 7fa51d13d700 1 -- 172.21.15.7:0/1318824170 >> 172.21.15.7:0/1318824170 conn(0x7fa5180792d0 msgr2=0x7fa518100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:41.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.635+0000 7fa51d13d700 1 -- 172.21.15.7:0/1318824170 shutdown_connections 2024-03-23T13:18:41.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.635+0000 7fa51d13d700 1 -- 172.21.15.7:0/1318824170 wait complete. 2024-03-23T13:18:41.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:41 smithi007 ceph-mon[32184]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:41.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:41 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:18:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:41 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1318824170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T13:18:41.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.820+0000 7f84dd463700 1 -- 172.21.15.7:0/2218256128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f84d81a7300 con 0x7f84d8101aa0 2024-03-23T13:18:41.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.820+0000 7f84bffff700 1 -- 172.21.15.7:0/2218256128 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f84d81a7300 con 0x7f84d8101aa0 2024-03-23T13:18:41.822 INFO:teuthology.orchestra.run.smithi007.stdout:107374182464 2024-03-23T13:18:41.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 -- 172.21.15.7:0/2218256128 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84c00a2260 msgr2=0x7f84c00a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:41.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84c00a2260 0x7f84c00a4720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009fd0 tx=0x7f84c8009260 comp rx=0 tx=0).stop 2024-03-23T13:18:41.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 -- 172.21.15.7:0/2218256128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8101aa0 msgr2=0x7f84d81032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8101aa0 0x7f84d81032d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e9a0 tx=0x7f84cc00ecb0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 -- 172.21.15.7:0/2218256128 shutdown_connections 2024-03-23T13:18:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84c00a2260 0x7f84c00a4720 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d8107190 0x7f84d8103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84d81010e0 0x7f84d8102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 --2- 172.21.15.7:0/2218256128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8101aa0 0x7f84d81032d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 -- 172.21.15.7:0/2218256128 >> 172.21.15.7:0/2218256128 conn(0x7f84d8078110 msgr2=0x7f84d8100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 -- 172.21.15.7:0/2218256128 shutdown_connections 2024-03-23T13:18:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:41.823+0000 7f84dd463700 1 -- 172.21.15.7:0/2218256128 wait complete. 2024-03-23T13:18:41.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:41 smithi130 ceph-mon[36225]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:41.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:41 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:18:41.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:41 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1318824170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T13:18:42.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:41 smithi155 ceph-mon[36157]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:18:42.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:41 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:18:42.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:41 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1318824170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T13:18:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.733+0000 7f786cd8e700 1 -- 172.21.15.7:0/1748582163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f786806b310 msgr2=0x7f786810f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.733+0000 7f786cd8e700 1 --2- 172.21.15.7:0/1748582163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f786806b310 0x7f786810f050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f785800b0d0 tx=0x7f785800b3e0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.733+0000 7f786cd8e700 1 -- 172.21.15.7:0/1748582163 shutdown_connections 2024-03-23T13:18:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.733+0000 7f786cd8e700 1 --2- 172.21.15.7:0/1748582163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f786806b310 0x7f786810f050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.733+0000 7f786cd8e700 1 --2- 172.21.15.7:0/1748582163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7868108230 0x7f7868108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.733+0000 7f786cd8e700 1 --2- 172.21.15.7:0/1748582163 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f786806a950 0x7f786806add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.733+0000 7f786cd8e700 1 -- 172.21.15.7:0/1748582163 >> 172.21.15.7:0/1748582163 conn(0x7f7868078110 msgr2=0x7f7868078520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.733+0000 7f786cd8e700 1 -- 172.21.15.7:0/1748582163 shutdown_connections 2024-03-23T13:18:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.733+0000 7f786cd8e700 1 -- 172.21.15.7:0/1748582163 wait complete. 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786cd8e700 1 Processor -- start 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786cd8e700 1 -- start start 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786cd8e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f786806a950 0x7f78681011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786cd8e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f786806b310 0x7f78681016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786cd8e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7868108230 0x7f7868101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786cd8e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78681a6b60 con 0x7f7868108230 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786cd8e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f78681a6cd0 con 0x7f786806a950 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786cd8e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78681a6e90 con 0x7f786806b310 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f786806a950 0x7f78681011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f7866d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7868108230 0x7f7868101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786659c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f786806a950 0x7f78681011b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:50402/0 (socket says 172.21.15.7:50402) 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786659c700 1 -- 172.21.15.7:0/2843318347 learned_addr learned my addr 172.21.15.7:0/2843318347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f7865d9b700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f786806b310 0x7f78681016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786659c700 1 -- 172.21.15.7:0/2843318347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f786806b310 msgr2=0x7f78681016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786659c700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f786806b310 0x7f78681016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786659c700 1 -- 172.21.15.7:0/2843318347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7868108230 msgr2=0x7f7868101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786659c700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7868108230 0x7f7868101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.734+0000 7f786659c700 1 -- 172.21.15.7:0/2843318347 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7858009d00 con 0x7f786806a950 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.735+0000 7f786659c700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f786806a950 0x7f78681011b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f786000e9a0 tx=0x7f786000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.735+0000 7f785f7fe700 1 -- 172.21.15.7:0/2843318347 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860009790 con 0x7f786806a950 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.735+0000 7f785f7fe700 1 -- 172.21.15.7:0/2843318347 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7860010430 con 0x7f786806a950 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.735+0000 7f786cd8e700 1 -- 172.21.15.7:0/2843318347 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78681a7170 con 0x7f786806a950 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.735+0000 7f785f7fe700 1 -- 172.21.15.7:0/2843318347 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860018680 con 0x7f786806a950 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.735+0000 7f786cd8e700 1 -- 172.21.15.7:0/2843318347 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78681a7670 con 0x7f786806a950 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.736+0000 7f786cd8e700 1 -- 172.21.15.7:0/2843318347 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7868068ec0 con 0x7f786806a950 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.737+0000 7f785f7fe700 1 -- 172.21.15.7:0/2843318347 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f7860010910 con 0x7f786806a950 2024-03-23T13:18:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.737+0000 7f785f7fe700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78540a21e0 0x7f78540a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.737+0000 7f7865d9b700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78540a21e0 0x7f78540a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.738+0000 7f785f7fe700 1 -- 172.21.15.7:0/2843318347 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8211+0+0 (secure 0 0 0) 0x7f7860014070 con 0x7f786806a950 2024-03-23T13:18:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.738+0000 7f7865d9b700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78540a21e0 0x7f78540a46a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7868106fb0 tx=0x7f7850009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.741+0000 7f785f7fe700 1 -- 172.21.15.7:0/2843318347 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7860096e30 con 0x7f786806a950 2024-03-23T13:18:42.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:42 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2218256128' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T13:18:42.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-23T13:18:42.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-23T13:18:42.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-03-23T13:18:42.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-23T13:18:42.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-03-23T13:18:42.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-23T13:18:42.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-23T13:18:42.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:42 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-23T13:18:42.870 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332140 for osd.7 2024-03-23T13:18:42.870 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:42.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:42 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2218256128' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T13:18:42.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-23T13:18:42.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-23T13:18:42.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-03-23T13:18:42.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-23T13:18:42.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-03-23T13:18:42.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-23T13:18:42.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-23T13:18:42.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:42 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-23T13:18:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.946+0000 7f786cd8e700 1 -- 172.21.15.7:0/2843318347 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7868112300 con 0x7f786806a950 2024-03-23T13:18:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.946+0000 7f785f7fe700 1 -- 172.21.15.7:0/2843318347 <== mon.2 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f78600105a0 con 0x7f786806a950 2024-03-23T13:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout:141733920828 2024-03-23T13:18:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 -- 172.21.15.7:0/2843318347 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78540a21e0 msgr2=0x7f78540a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78540a21e0 0x7f78540a46a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7868106fb0 tx=0x7f7850009260 comp rx=0 tx=0).stop 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 -- 172.21.15.7:0/2843318347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f786806a950 msgr2=0x7f78681011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f786806a950 0x7f78681011b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f786000e9a0 tx=0x7f786000ecb0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 -- 172.21.15.7:0/2843318347 shutdown_connections 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f78540a21e0 0x7f78540a46a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7868108230 0x7f7868101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f786806a950 0x7f78681011b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 --2- 172.21.15.7:0/2843318347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f786806b310 0x7f78681016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 -- 172.21.15.7:0/2843318347 >> 172.21.15.7:0/2843318347 conn(0x7f7868078110 msgr2=0x7f78680ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 -- 172.21.15.7:0/2843318347 shutdown_connections 2024-03-23T13:18:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:42.949+0000 7f786cd8e700 1 -- 172.21.15.7:0/2843318347 wait complete. 2024-03-23T13:18:43.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:42 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2218256128' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T13:18:43.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-23T13:18:43.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-23T13:18:43.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-03-23T13:18:43.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-23T13:18:43.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-03-23T13:18:43.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-23T13:18:43.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-23T13:18:43.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:42 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-23T13:18:43.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:18:43.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-23T13:18:43.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-23T13:18:43.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-03-23T13:18:43.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-23T13:18:43.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-03-23T13:18:43.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-23T13:18:43.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-23T13:18:43.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-23T13:18:43.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T13:18:43.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2843318347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T13:18:43.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:43.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:43.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:43.946 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:43 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:43.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:18:43.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-23T13:18:43.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-23T13:18:43.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-03-23T13:18:43.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-23T13:18:43.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-03-23T13:18:43.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-23T13:18:43.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-23T13:18:43.948 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-23T13:18:43.949 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T13:18:43.949 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2843318347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T13:18:43.949 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:43.949 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:43.949 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:43.949 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:43 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:44.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2843318347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:43 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:44.107 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808863 got 287762808867 for osd.9 2024-03-23T13:18:44.107 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:44.736 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182464 for osd.3 2024-03-23T13:18:44.737 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:44.940 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:44 smithi007 ceph-mon[32184]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T13:18:44.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:44 smithi130 ceph-mon[36225]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T13:18:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.943+0000 7feeb7168700 1 -- 172.21.15.7:0/3256076820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb01007f0 msgr2=0x7feeb0100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.943+0000 7feeb7168700 1 --2- 172.21.15.7:0/3256076820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb01007f0 0x7feeb0100c50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009990 tx=0x7fee9c009ca0 comp rx=0 tx=0).stop 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.943+0000 7feeb7168700 1 -- 172.21.15.7:0/3256076820 shutdown_connections 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.943+0000 7feeb7168700 1 --2- 172.21.15.7:0/3256076820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb0101380 0x7feeb0110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.943+0000 7feeb7168700 1 --2- 172.21.15.7:0/3256076820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feeb0103140 0x7feeb0103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.943+0000 7feeb7168700 1 --2- 172.21.15.7:0/3256076820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb01007f0 0x7feeb0100c50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.943+0000 7feeb7168700 1 -- 172.21.15.7:0/3256076820 >> 172.21.15.7:0/3256076820 conn(0x7feeb0077700 msgr2=0x7feeb00feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb7168700 1 -- 172.21.15.7:0/3256076820 shutdown_connections 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb7168700 1 -- 172.21.15.7:0/3256076820 wait complete. 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb7168700 1 Processor -- start 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb7168700 1 -- start start 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb7168700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb01007f0 0x7feeb01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb7168700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb0101380 0x7feeb01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb7168700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feeb0103140 0x7feeb019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb7168700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feeb019ae80 con 0x7feeb0101380 2024-03-23T13:18:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb7168700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7feeb019aff0 con 0x7feeb0103140 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb7168700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feeb019b1b0 con 0x7feeb01007f0 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeaffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb0101380 0x7feeb01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeaffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb0101380 0x7feeb01a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58660/0 (socket says 172.21.15.7:58660) 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeaffff700 1 -- 172.21.15.7:0/813127407 learned_addr learned my addr 172.21.15.7:0/813127407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb4f04700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb01007f0 0x7feeb01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.944+0000 7feeb5705700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feeb0103140 0x7feeb019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeaffff700 1 -- 172.21.15.7:0/813127407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb01007f0 msgr2=0x7feeb01a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeaffff700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb01007f0 0x7feeb01a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeaffff700 1 -- 172.21.15.7:0/813127407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feeb0103140 msgr2=0x7feeb019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeaffff700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feeb0103140 0x7feeb019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeaffff700 1 -- 172.21.15.7:0/813127407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c0095f0 con 0x7feeb0101380 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeaffff700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb0101380 0x7feeb01a0b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00f690 tx=0x7fee9c00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeadffb700 1 -- 172.21.15.7:0/813127407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c01d070 con 0x7feeb0101380 2024-03-23T13:18:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeadffb700 1 -- 172.21.15.7:0/813127407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9c003680 con 0x7feeb0101380 2024-03-23T13:18:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeb7168700 1 -- 172.21.15.7:0/813127407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb019b430 con 0x7feeb0101380 2024-03-23T13:18:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeadffb700 1 -- 172.21.15.7:0/813127407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c022e10 con 0x7feeb0101380 2024-03-23T13:18:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.945+0000 7feeb7168700 1 -- 172.21.15.7:0/813127407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb01a7680 con 0x7feeb0101380 2024-03-23T13:18:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.946+0000 7feeb7168700 1 -- 172.21.15.7:0/813127407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb010c6e0 con 0x7feeb0101380 2024-03-23T13:18:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.947+0000 7feeadffb700 1 -- 172.21.15.7:0/813127407 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fee9c00fbc0 con 0x7feeb0101380 2024-03-23T13:18:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.948+0000 7feeadffb700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7feea00a2260 0x7feea00a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.948+0000 7feeb4f04700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7feea00a2260 0x7feea00a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.948+0000 7feeadffb700 1 -- 172.21.15.7:0/813127407 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fee9c0cf050 con 0x7feeb0101380 2024-03-23T13:18:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.948+0000 7feeb4f04700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7feea00a2260 0x7feea00a4720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feea4007b60 tx=0x7feea40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:44.952+0000 7feeadffb700 1 -- 172.21.15.7:0/813127407 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fee9c098020 con 0x7feeb0101380 2024-03-23T13:18:45.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:44 smithi155 ceph-mon[36157]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T13:18:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.160+0000 7feeb7168700 1 -- 172.21.15.7:0/813127407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7feeb019bd00 con 0x7feeb0101380 2024-03-23T13:18:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.160+0000 7feeadffb700 1 -- 172.21.15.7:0/813127407 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fee9c09b6e0 con 0x7feeb0101380 2024-03-23T13:18:45.161 INFO:teuthology.orchestra.run.smithi007.stdout:343597383704 2024-03-23T13:18:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.162+0000 7feeb7168700 1 -- 172.21.15.7:0/813127407 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7feea00a2260 msgr2=0x7feea00a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.162+0000 7feeb7168700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7feea00a2260 0x7feea00a4720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feea4007b60 tx=0x7feea40073d0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.162+0000 7feeb7168700 1 -- 172.21.15.7:0/813127407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb0101380 msgr2=0x7feeb01a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.162+0000 7feeb7168700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb0101380 0x7feeb01a0b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00f690 tx=0x7fee9c00f6c0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.162+0000 7feeb7168700 1 -- 172.21.15.7:0/813127407 shutdown_connections 2024-03-23T13:18:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.162+0000 7feeb7168700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7feea00a2260 0x7feea00a4720 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.162+0000 7feeb7168700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feeb0103140 0x7feeb019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.162+0000 7feeb7168700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb01007f0 0x7feeb01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.162+0000 7feeb7168700 1 --2- 172.21.15.7:0/813127407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb0101380 0x7feeb01a0b90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.162+0000 7feeb7168700 1 -- 172.21.15.7:0/813127407 >> 172.21.15.7:0/813127407 conn(0x7feeb0077700 msgr2=0x7feeb00fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:45.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.163+0000 7feeb7168700 1 -- 172.21.15.7:0/813127407 shutdown_connections 2024-03-23T13:18:45.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.163+0000 7feeb7168700 1 -- 172.21.15.7:0/813127407 wait complete. 2024-03-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.569+0000 7f1363de8700 1 -- 172.21.15.7:0/1219185983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f135c101390 msgr2=0x7f135c101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.569+0000 7f1363de8700 1 --2- 172.21.15.7:0/1219185983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f135c101390 0x7f135c101770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f134c009990 tx=0x7f134c009ca0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.569+0000 7f1363de8700 1 -- 172.21.15.7:0/1219185983 shutdown_connections 2024-03-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.569+0000 7f1363de8700 1 --2- 172.21.15.7:0/1219185983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f135c1063e0 0x7f135c10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.569+0000 7f1363de8700 1 --2- 172.21.15.7:0/1219185983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f135c101390 0x7f135c101770 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.569+0000 7f1363de8700 1 --2- 172.21.15.7:0/1219185983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f135c101d40 0x7f135c105d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.569+0000 7f1363de8700 1 -- 172.21.15.7:0/1219185983 >> 172.21.15.7:0/1219185983 conn(0x7f135c077150 msgr2=0x7f135c077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.569+0000 7f1363de8700 1 -- 172.21.15.7:0/1219185983 shutdown_connections 2024-03-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.569+0000 7f1363de8700 1 -- 172.21.15.7:0/1219185983 wait complete. 2024-03-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1363de8700 1 Processor -- start 2024-03-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1363de8700 1 -- start start 2024-03-23T13:18:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1363de8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f135c101390 0x7f135c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1363de8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f135c101d40 0x7f135c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1363de8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f135c1063e0 0x7f135c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1363de8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f135c19aef0 con 0x7f135c101d40 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1363de8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f135c19b060 con 0x7f135c101390 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1363de8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f135c19b220 con 0x7f135c1063e0 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1361383700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f135c101d40 0x7f135c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1361383700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f135c101d40 0x7f135c1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58672/0 (socket says 172.21.15.7:58672) 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1361383700 1 -- 172.21.15.7:0/1979784069 learned_addr learned my addr 172.21.15.7:0/1979784069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1361b84700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f135c101390 0x7f135c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.570+0000 7f1362385700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f135c1063e0 0x7f135c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1361383700 1 -- 172.21.15.7:0/1979784069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f135c1063e0 msgr2=0x7f135c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1361383700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f135c1063e0 0x7f135c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1361383700 1 -- 172.21.15.7:0/1979784069 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f135c101390 msgr2=0x7f135c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1361383700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f135c101390 0x7f135c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1361383700 1 -- 172.21.15.7:0/1979784069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c0095f0 con 0x7f135c101d40 2024-03-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1361383700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f135c101d40 0x7f135c1a0bb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f135800d950 tx=0x7f135800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1352ffd700 1 -- 172.21.15.7:0/1979784069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358004220 con 0x7f135c101d40 2024-03-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1352ffd700 1 -- 172.21.15.7:0/1979784069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1358007620 con 0x7f135c101d40 2024-03-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1363de8700 1 -- 172.21.15.7:0/1979784069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c19b500 con 0x7f135c101d40 2024-03-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1352ffd700 1 -- 172.21.15.7:0/1979784069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13580056e0 con 0x7f135c101d40 2024-03-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.571+0000 7f1363de8700 1 -- 172.21.15.7:0/1979784069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c1a75f0 con 0x7f135c101d40 2024-03-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.572+0000 7f1363de8700 1 -- 172.21.15.7:0/1979784069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135c1027e0 con 0x7f135c101d40 2024-03-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.574+0000 7f1352ffd700 1 -- 172.21.15.7:0/1979784069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f1358004380 con 0x7f135c101d40 2024-03-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.574+0000 7f1352ffd700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f13480a21e0 0x7f13480a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.574+0000 7f1361b84700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f13480a21e0 0x7f13480a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.574+0000 7f1352ffd700 1 -- 172.21.15.7:0/1979784069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f13580ce090 con 0x7f135c101d40 2024-03-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.575+0000 7f1361b84700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f13480a21e0 0x7f13480a46a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f134c00b2e0 tx=0x7f134c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:45.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.578+0000 7f1352ffd700 1 -- 172.21.15.7:0/1979784069 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1358096f10 con 0x7f135c101d40 2024-03-23T13:18:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:45 smithi007 ceph-mon[32184]: pgmap v318: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:18:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:45 smithi007 ceph-mon[32184]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-23T13:18:45.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:45 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/813127407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T13:18:45.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.796+0000 7f1363de8700 1 -- 172.21.15.7:0/1979784069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f135c111130 con 0x7f135c101d40 2024-03-23T13:18:45.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.797+0000 7f1352ffd700 1 -- 172.21.15.7:0/1979784069 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1358016090 con 0x7f135c101d40 2024-03-23T13:18:45.798 INFO:teuthology.orchestra.run.smithi007.stdout:253403070505 2024-03-23T13:18:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 -- 172.21.15.7:0/1979784069 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f13480a21e0 msgr2=0x7f13480a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f13480a21e0 0x7f13480a46a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f134c00b2e0 tx=0x7f134c005960 comp rx=0 tx=0).stop 2024-03-23T13:18:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 -- 172.21.15.7:0/1979784069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f135c101d40 msgr2=0x7f135c1a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f135c101d40 0x7f135c1a0bb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f135800d950 tx=0x7f135800dc60 comp rx=0 tx=0).stop 2024-03-23T13:18:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 -- 172.21.15.7:0/1979784069 shutdown_connections 2024-03-23T13:18:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f13480a21e0 0x7f13480a46a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f135c1063e0 0x7f135c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f135c101390 0x7f135c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 --2- 172.21.15.7:0/1979784069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f135c101d40 0x7f135c1a0bb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 -- 172.21.15.7:0/1979784069 >> 172.21.15.7:0/1979784069 conn(0x7f135c077150 msgr2=0x7f135c107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 -- 172.21.15.7:0/1979784069 shutdown_connections 2024-03-23T13:18:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:45.799+0000 7f1363de8700 1 -- 172.21.15.7:0/1979784069 wait complete. 2024-03-23T13:18:45.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:45 smithi130 ceph-mon[36225]: pgmap v318: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:18:45.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:45 smithi130 ceph-mon[36225]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-23T13:18:45.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:45 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/813127407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T13:18:46.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:45 smithi155 ceph-mon[36157]: pgmap v318: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:18:46.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:45 smithi155 ceph-mon[36157]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-23T13:18:46.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:45 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/813127407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T13:18:46.945 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:46 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1979784069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T13:18:47.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:46 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1979784069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T13:18:47.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:46 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1979784069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T13:18:47.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:47 smithi155 ceph-mon[36157]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 13 B/s, 0 objects/s recovering 2024-03-23T13:18:47.944 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:47 smithi130 ceph-mon[36225]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 13 B/s, 0 objects/s recovering 2024-03-23T13:18:48.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:47 smithi007 ceph-mon[32184]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 13 B/s, 0 objects/s recovering 2024-03-23T13:18:49.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:48 smithi007 ceph-mon[32184]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-23T13:18:49.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:48 smithi007 ceph-mon[32184]: Cluster is now healthy 2024-03-23T13:18:49.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:48 smithi155 ceph-mon[36157]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-23T13:18:49.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:48 smithi155 ceph-mon[36157]: Cluster is now healthy 2024-03-23T13:18:49.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:48 smithi130 ceph-mon[36225]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-23T13:18:49.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:48 smithi130 ceph-mon[36225]: Cluster is now healthy 2024-03-23T13:18:49.225 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070501 got 253403070505 for osd.8 2024-03-23T13:18:49.226 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:49.811 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383700 got 343597383704 for osd.11 2024-03-23T13:18:49.811 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:50.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:49 smithi155 ceph-mon[36157]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T13:18:50.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:49 smithi007 ceph-mon[32184]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T13:18:50.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:49 smithi130 ceph-mon[36225]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T13:18:51.065 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920822 got 141733920828 for osd.4 2024-03-23T13:18:51.065 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:51.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.259+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2413391213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868107140 msgr2=0x7fb868107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:51.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.259+0000 7fb86d6bb700 1 --2- 172.21.15.7:0/2413391213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868107140 0x7fb868107520 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb858009990 tx=0x7fb858009ca0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.260+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2413391213 shutdown_connections 2024-03-23T13:18:51.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.260+0000 7fb86d6bb700 1 --2- 172.21.15.7:0/2413391213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb868101aa0 0x7fb86810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.260+0000 7fb86d6bb700 1 --2- 172.21.15.7:0/2413391213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868107140 0x7fb868107520 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.260+0000 7fb86d6bb700 1 --2- 172.21.15.7:0/2413391213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb868101100 0x7fb868101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.260+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2413391213 >> 172.21.15.7:0/2413391213 conn(0x7fb868078030 msgr2=0x7fb868078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:51.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.260+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2413391213 shutdown_connections 2024-03-23T13:18:51.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.260+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2413391213 wait complete. 2024-03-23T13:18:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.260+0000 7fb86d6bb700 1 Processor -- start 2024-03-23T13:18:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb86d6bb700 1 -- start start 2024-03-23T13:18:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb86d6bb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb868101100 0x7fb868198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb86d6bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868101aa0 0x7fb868198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb86d6bb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb868107140 0x7fb86819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb86d6bb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb86819d590 con 0x7fb868101aa0 2024-03-23T13:18:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb86d6bb700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb86819d700 con 0x7fb868107140 2024-03-23T13:18:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb86d6bb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb86819d8c0 con 0x7fb868101100 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8667fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868101aa0 0x7fb868198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8667fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868101aa0 0x7fb868198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58688/0 (socket says 172.21.15.7:58688) 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8667fc700 1 -- 172.21.15.7:0/2747231988 learned_addr learned my addr 172.21.15.7:0/2747231988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb866ffd700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb868101100 0x7fb868198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8677fe700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb868107140 0x7fb86819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8667fc700 1 -- 172.21.15.7:0/2747231988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb868101100 msgr2=0x7fb868198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8667fc700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb868101100 0x7fb868198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8667fc700 1 -- 172.21.15.7:0/2747231988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb868107140 msgr2=0x7fb86819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8667fc700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb868107140 0x7fb86819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8667fc700 1 -- 172.21.15.7:0/2747231988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8580095f0 con 0x7fb868101aa0 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8677fe700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb868107140 0x7fb86819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.261+0000 7fb8667fc700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868101aa0 0x7fb868198c40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9a0 tx=0x7fb85c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.262+0000 7fb84ffff700 1 -- 172.21.15.7:0/2747231988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c009790 con 0x7fb868101aa0 2024-03-23T13:18:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.262+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2747231988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86819dba0 con 0x7fb868101aa0 2024-03-23T13:18:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.262+0000 7fb84ffff700 1 -- 172.21.15.7:0/2747231988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c010430 con 0x7fb868101aa0 2024-03-23T13:18:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.262+0000 7fb84ffff700 1 -- 172.21.15.7:0/2747231988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c018680 con 0x7fb868101aa0 2024-03-23T13:18:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.262+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2747231988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8681a5400 con 0x7fb868101aa0 2024-03-23T13:18:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.263+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2747231988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb868068ec0 con 0x7fb868101aa0 2024-03-23T13:18:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.264+0000 7fb84ffff700 1 -- 172.21.15.7:0/2747231988 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fb85c010910 con 0x7fb868101aa0 2024-03-23T13:18:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.264+0000 7fb84ffff700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb8500a21e0 0x7fb8500a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.264+0000 7fb866ffd700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb8500a21e0 0x7fb8500a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.264+0000 7fb84ffff700 1 -- 172.21.15.7:0/2747231988 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb85c014070 con 0x7fb868101aa0 2024-03-23T13:18:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.265+0000 7fb866ffd700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb8500a21e0 0x7fb8500a46a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb85800b2e0 tx=0x7fb858005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.269+0000 7fb84ffff700 1 -- 172.21.15.7:0/2747231988 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb85c053280 con 0x7fb868101aa0 2024-03-23T13:18:51.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.476+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2747231988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb868111100 con 0x7fb868101aa0 2024-03-23T13:18:51.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.476+0000 7fb84ffff700 1 -- 172.21.15.7:0/2747231988 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb85c096ed0 con 0x7fb868101aa0 2024-03-23T13:18:51.477 INFO:teuthology.orchestra.run.smithi007.stdout:309237645343 2024-03-23T13:18:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.478+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2747231988 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb8500a21e0 msgr2=0x7fb8500a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.478+0000 7fb86d6bb700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb8500a21e0 0x7fb8500a46a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb85800b2e0 tx=0x7fb858005960 comp rx=0 tx=0).stop 2024-03-23T13:18:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.478+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2747231988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868101aa0 msgr2=0x7fb868198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.478+0000 7fb86d6bb700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868101aa0 0x7fb868198c40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9a0 tx=0x7fb85c00ecb0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.478+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2747231988 shutdown_connections 2024-03-23T13:18:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.478+0000 7fb86d6bb700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb8500a21e0 0x7fb8500a46a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.478+0000 7fb86d6bb700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb868107140 0x7fb86819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.478+0000 7fb86d6bb700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb868101100 0x7fb868198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.478+0000 7fb86d6bb700 1 --2- 172.21.15.7:0/2747231988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868101aa0 0x7fb868198c40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.478+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2747231988 >> 172.21.15.7:0/2747231988 conn(0x7fb868078030 msgr2=0x7fb868100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.479+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2747231988 shutdown_connections 2024-03-23T13:18:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.479+0000 7fb86d6bb700 1 -- 172.21.15.7:0/2747231988 wait complete. 2024-03-23T13:18:51.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.842+0000 7f7ad2d23700 1 -- 172.21.15.7:0/845225955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7acc106480 msgr2=0x7f7acc10e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:51.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.842+0000 7f7ad2d23700 1 --2- 172.21.15.7:0/845225955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7acc106480 0x7f7acc10e050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400b0a0 tx=0x7f7ac400b3b0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.842+0000 7f7ad2d23700 1 -- 172.21.15.7:0/845225955 shutdown_connections 2024-03-23T13:18:51.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.842+0000 7f7ad2d23700 1 --2- 172.21.15.7:0/845225955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7acc106480 0x7f7acc10e050 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.842+0000 7f7ad2d23700 1 --2- 172.21.15.7:0/845225955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7acc101b60 0x7f7acc101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.842+0000 7f7ad2d23700 1 --2- 172.21.15.7:0/845225955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7acc102510 0x7f7acc105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.842+0000 7f7ad2d23700 1 -- 172.21.15.7:0/845225955 >> 172.21.15.7:0/845225955 conn(0x7f7acc077700 msgr2=0x7f7acc0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:51.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.842+0000 7f7ad2d23700 1 -- 172.21.15.7:0/845225955 shutdown_connections 2024-03-23T13:18:51.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.842+0000 7f7ad2d23700 1 -- 172.21.15.7:0/845225955 wait complete. 2024-03-23T13:18:51.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad2d23700 1 Processor -- start 2024-03-23T13:18:51.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad2d23700 1 -- start start 2024-03-23T13:18:51.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad2d23700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7acc101b60 0x7f7acc197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:51.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad2d23700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7acc102510 0x7f7acc197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:51.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad2d23700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7acc106480 0x7f7acc197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:51.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad2d23700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc1a6c80 con 0x7f7acc102510 2024-03-23T13:18:51.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad2d23700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc1a6dc0 con 0x7f7acc101b60 2024-03-23T13:18:51.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad2d23700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc1a6f80 con 0x7f7acc106480 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7acbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7acc102510 0x7f7acc197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7acbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7acc102510 0x7f7acc197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58708/0 (socket says 172.21.15.7:58708) 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad0abf700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7acc101b60 0x7f7acc197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7acbfff700 1 -- 172.21.15.7:0/3840911285 learned_addr learned my addr 172.21.15.7:0/3840911285 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad12c0700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7acc106480 0x7f7acc197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad0abf700 1 -- 172.21.15.7:0/3840911285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7acc106480 msgr2=0x7f7acc197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad0abf700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7acc106480 0x7f7acc197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad0abf700 1 -- 172.21.15.7:0/3840911285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7acc102510 msgr2=0x7f7acc197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad0abf700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7acc102510 0x7f7acc197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7ad0abf700 1 -- 172.21.15.7:0/3840911285 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac4009d00 con 0x7f7acc101b60 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.843+0000 7f7acbfff700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7acc102510 0x7f7acc197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.844+0000 7f7ad0abf700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7acc101b60 0x7f7acc197570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000ea20 tx=0x7f7ac000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.844+0000 7f7ac9ffb700 1 -- 172.21.15.7:0/3840911285 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac000cbd0 con 0x7f7acc101b60 2024-03-23T13:18:51.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.844+0000 7f7ac9ffb700 1 -- 172.21.15.7:0/3840911285 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ac000ee60 con 0x7f7acc101b60 2024-03-23T13:18:51.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.844+0000 7f7ad2d23700 1 -- 172.21.15.7:0/3840911285 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7acc1a7260 con 0x7f7acc101b60 2024-03-23T13:18:51.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.844+0000 7f7ac9ffb700 1 -- 172.21.15.7:0/3840911285 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0018680 con 0x7f7acc101b60 2024-03-23T13:18:51.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.844+0000 7f7ad2d23700 1 -- 172.21.15.7:0/3840911285 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7acc1a7760 con 0x7f7acc101b60 2024-03-23T13:18:51.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.845+0000 7f7ad2d23700 1 -- 172.21.15.7:0/3840911285 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7acc1097a0 con 0x7f7acc101b60 2024-03-23T13:18:51.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.846+0000 7f7ac9ffb700 1 -- 172.21.15.7:0/3840911285 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f7ac00187e0 con 0x7f7acc101b60 2024-03-23T13:18:51.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.847+0000 7f7ac9ffb700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7abc0a2260 0x7f7abc0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:51.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.847+0000 7f7ac9ffb700 1 -- 172.21.15.7:0/3840911285 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7ac0014070 con 0x7f7acc101b60 2024-03-23T13:18:51.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.847+0000 7f7acbfff700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7abc0a2260 0x7f7abc0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:51.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.848+0000 7f7acbfff700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7abc0a2260 0x7f7abc0a4720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7acc1996d0 tx=0x7f7ab8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:51.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:51.852+0000 7f7ac9ffb700 1 -- 172.21.15.7:0/3840911285 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7ac0053280 con 0x7f7acc101b60 2024-03-23T13:18:51.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:51 smithi007 ceph-mon[32184]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T13:18:51.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:51 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2747231988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T13:18:52.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:51 smithi155 ceph-mon[36157]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T13:18:52.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:51 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2747231988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T13:18:52.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.060+0000 7f7ad2d23700 1 -- 172.21.15.7:0/3840911285 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7acc1028f0 con 0x7f7acc101b60 2024-03-23T13:18:52.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.061+0000 7f7ac9ffb700 1 -- 172.21.15.7:0/3840911285 <== mon.2 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7ac0096df0 con 0x7f7acc101b60 2024-03-23T13:18:52.061 INFO:teuthology.orchestra.run.smithi007.stdout:77309411400 2024-03-23T13:18:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 -- 172.21.15.7:0/3840911285 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7abc0a2260 msgr2=0x7f7abc0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7abc0a2260 0x7f7abc0a4720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7acc1996d0 tx=0x7f7ab8009260 comp rx=0 tx=0).stop 2024-03-23T13:18:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 -- 172.21.15.7:0/3840911285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7acc101b60 msgr2=0x7f7acc197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7acc101b60 0x7f7acc197570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000ea20 tx=0x7f7ac000ed30 comp rx=0 tx=0).stop 2024-03-23T13:18:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 -- 172.21.15.7:0/3840911285 shutdown_connections 2024-03-23T13:18:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7abc0a2260 0x7f7abc0a4720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7acc106480 0x7f7acc197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7acc101b60 0x7f7acc197570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 --2- 172.21.15.7:0/3840911285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7acc102510 0x7f7acc197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 -- 172.21.15.7:0/3840911285 >> 172.21.15.7:0/3840911285 conn(0x7f7acc077700 msgr2=0x7f7acc104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 -- 172.21.15.7:0/3840911285 shutdown_connections 2024-03-23T13:18:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.063+0000 7f7ad2d23700 1 -- 172.21.15.7:0/3840911285 wait complete. 2024-03-23T13:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.067+0000 7fdaafcd2700 1 -- 172.21.15.7:0/4109124583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa8107580 msgr2=0x7fdaa810f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.067+0000 7fdaafcd2700 1 --2- 172.21.15.7:0/4109124583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa8107580 0x7fdaa810f100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b0a0 tx=0x7fdaa000b3b0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.067+0000 7fdaafcd2700 1 -- 172.21.15.7:0/4109124583 shutdown_connections 2024-03-23T13:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.067+0000 7fdaafcd2700 1 --2- 172.21.15.7:0/4109124583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa8107580 0x7fdaa810f100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.067+0000 7fdaafcd2700 1 --2- 172.21.15.7:0/4109124583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa806a9a0 0x7fdaa806ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.067+0000 7fdaafcd2700 1 --2- 172.21.15.7:0/4109124583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaa806b350 0x7fdaa8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.067+0000 7fdaafcd2700 1 -- 172.21.15.7:0/4109124583 >> 172.21.15.7:0/4109124583 conn(0x7fdaa8077150 msgr2=0x7fdaa8077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.068+0000 7fdaafcd2700 1 -- 172.21.15.7:0/4109124583 shutdown_connections 2024-03-23T13:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.068+0000 7fdaafcd2700 1 -- 172.21.15.7:0/4109124583 wait complete. 2024-03-23T13:18:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.068+0000 7fdaafcd2700 1 Processor -- start 2024-03-23T13:18:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.068+0000 7fdaafcd2700 1 -- start start 2024-03-23T13:18:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaafcd2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaa806a9a0 0x7fdaa819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaafcd2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa806b350 0x7fdaa819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaafcd2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa8107580 0x7fdaa81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaafcd2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8198e80 con 0x7fdaa806b350 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaafcd2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8198ff0 con 0x7fdaa806a9a0 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaafcd2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81991b0 con 0x7fdaa8107580 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaad26d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa806b350 0x7fdaa819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaad26d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa806b350 0x7fdaa819eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58734/0 (socket says 172.21.15.7:58734) 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaad26d700 1 -- 172.21.15.7:0/2503081201 learned_addr learned my addr 172.21.15.7:0/2503081201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaada6e700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaa806a9a0 0x7fdaa819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaae26f700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa8107580 0x7fdaa81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaad26d700 1 -- 172.21.15.7:0/2503081201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa8107580 msgr2=0x7fdaa81986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaad26d700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa8107580 0x7fdaa81986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaad26d700 1 -- 172.21.15.7:0/2503081201 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaa806a9a0 msgr2=0x7fdaa819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaad26d700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaa806a9a0 0x7fdaa819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaad26d700 1 -- 172.21.15.7:0/2503081201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa0009d00 con 0x7fdaa806b350 2024-03-23T13:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaada6e700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaa806a9a0 0x7fdaa819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fdaad26d700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa806b350 0x7fdaa819eaf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fda9800e940 tx=0x7fda9800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.069+0000 7fda9effd700 1 -- 172.21.15.7:0/2503081201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98009730 con 0x7fdaa806b350 2024-03-23T13:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.070+0000 7fda9effd700 1 -- 172.21.15.7:0/2503081201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda98010430 con 0x7fdaa806b350 2024-03-23T13:18:52.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.070+0000 7fdaafcd2700 1 -- 172.21.15.7:0/2503081201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa8199430 con 0x7fdaa806b350 2024-03-23T13:18:52.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.070+0000 7fda9effd700 1 -- 172.21.15.7:0/2503081201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98018660 con 0x7fdaa806b350 2024-03-23T13:18:52.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.070+0000 7fdaafcd2700 1 -- 172.21.15.7:0/2503081201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa8101300 con 0x7fdaa806b350 2024-03-23T13:18:52.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.071+0000 7fdaafcd2700 1 -- 172.21.15.7:0/2503081201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa810a770 con 0x7fdaa806b350 2024-03-23T13:18:52.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.072+0000 7fda9effd700 1 -- 172.21.15.7:0/2503081201 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fda980108a0 con 0x7fdaa806b350 2024-03-23T13:18:52.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.072+0000 7fda9effd700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fda940a21e0 0x7fda940a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:52.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.073+0000 7fdaada6e700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fda940a21e0 0x7fda940a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:52.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.073+0000 7fda9effd700 1 -- 172.21.15.7:0/2503081201 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fda98014070 con 0x7fdaa806b350 2024-03-23T13:18:52.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.073+0000 7fdaada6e700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fda940a21e0 0x7fda940a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4011490 tx=0x7fdaa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:52.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.077+0000 7fda9effd700 1 -- 172.21.15.7:0/2503081201 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fda98053280 con 0x7fdaa806b350 2024-03-23T13:18:52.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:51 smithi130 ceph-mon[36225]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T13:18:52.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:51 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2747231988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T13:18:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.297+0000 7fdaafcd2700 1 -- 172.21.15.7:0/2503081201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fdaa8199bb0 con 0x7fdaa806b350 2024-03-23T13:18:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.297+0000 7fda9effd700 1 -- 172.21.15.7:0/2503081201 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fda980d1030 con 0x7fdaa806b350 2024-03-23T13:18:52.297 INFO:teuthology.orchestra.run.smithi007.stdout:34359738450 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.299+0000 7fdaafcd2700 1 -- 172.21.15.7:0/2503081201 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fda940a21e0 msgr2=0x7fda940a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.299+0000 7fdaafcd2700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fda940a21e0 0x7fda940a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4011490 tx=0x7fdaa4009260 comp rx=0 tx=0).stop 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.299+0000 7fdaafcd2700 1 -- 172.21.15.7:0/2503081201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa806b350 msgr2=0x7fdaa819eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.299+0000 7fdaafcd2700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa806b350 0x7fdaa819eaf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fda9800e940 tx=0x7fda9800ec50 comp rx=0 tx=0).stop 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.299+0000 7fdaafcd2700 1 -- 172.21.15.7:0/2503081201 shutdown_connections 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.299+0000 7fdaafcd2700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fda940a21e0 0x7fda940a46a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.299+0000 7fdaafcd2700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa8107580 0x7fdaa81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.299+0000 7fdaafcd2700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaa806a9a0 0x7fdaa819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.299+0000 7fdaafcd2700 1 --2- 172.21.15.7:0/2503081201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa806b350 0x7fdaa819eaf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.299+0000 7fdaafcd2700 1 -- 172.21.15.7:0/2503081201 >> 172.21.15.7:0/2503081201 conn(0x7fdaa8077150 msgr2=0x7fdaa81008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.300+0000 7fdaafcd2700 1 -- 172.21.15.7:0/2503081201 shutdown_connections 2024-03-23T13:18:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:52.300+0000 7fdaafcd2700 1 -- 172.21.15.7:0/2503081201 wait complete. 2024-03-23T13:18:52.896 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:52 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3840911285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T13:18:52.896 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:52 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2503081201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T13:18:53.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:52 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3840911285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T13:18:53.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:52 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2503081201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T13:18:53.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:52 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3840911285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T13:18:53.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:52 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2503081201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T13:18:53.534 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645336 got 309237645343 for osd.10 2024-03-23T13:18:53.534 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.967+0000 7fb0292fe700 1 -- 172.21.15.7:0/805939107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024108340 msgr2=0x7fb024108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.967+0000 7fb0292fe700 1 --2- 172.21.15.7:0/805939107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024108340 0x7fb024108720 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb014009990 tx=0x7fb014009ca0 comp rx=0 tx=0).stop 2024-03-23T13:18:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.967+0000 7fb0292fe700 1 -- 172.21.15.7:0/805939107 shutdown_connections 2024-03-23T13:18:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.967+0000 7fb0292fe700 1 --2- 172.21.15.7:0/805939107 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb024100ce0 0x7fb02410f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.967+0000 7fb0292fe700 1 --2- 172.21.15.7:0/805939107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024108340 0x7fb024108720 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.967+0000 7fb0292fe700 1 --2- 172.21.15.7:0/805939107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb024100320 0x7fb0241007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.967+0000 7fb0292fe700 1 -- 172.21.15.7:0/805939107 >> 172.21.15.7:0/805939107 conn(0x7fb0240772d0 msgr2=0x7fb0240fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.968+0000 7fb0292fe700 1 -- 172.21.15.7:0/805939107 shutdown_connections 2024-03-23T13:18:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.968+0000 7fb0292fe700 1 -- 172.21.15.7:0/805939107 wait complete. 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.968+0000 7fb0292fe700 1 Processor -- start 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.968+0000 7fb0292fe700 1 -- start start 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.968+0000 7fb0292fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024100320 0x7fb024198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.968+0000 7fb0292fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb024100ce0 0x7fb024198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb0292fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb024108340 0x7fb02419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb0292fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb02419d550 con 0x7fb024100320 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb0292fe700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb02419d6c0 con 0x7fb024100ce0 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb0292fe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb02419d880 con 0x7fb024108340 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb022ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024100320 0x7fb024198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb022ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024100320 0x7fb024198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58754/0 (socket says 172.21.15.7:58754) 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb022ffd700 1 -- 172.21.15.7:0/2362402372 learned_addr learned my addr 172.21.15.7:0/2362402372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb0237fe700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb024108340 0x7fb02419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb0227fc700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb024100ce0 0x7fb024198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb022ffd700 1 -- 172.21.15.7:0/2362402372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb024108340 msgr2=0x7fb02419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb022ffd700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb024108340 0x7fb02419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb022ffd700 1 -- 172.21.15.7:0/2362402372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb024100ce0 msgr2=0x7fb024198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb022ffd700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb024100ce0 0x7fb024198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb022ffd700 1 -- 172.21.15.7:0/2362402372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0140095f0 con 0x7fb024100320 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb022ffd700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024100320 0x7fb024198670 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb014000c00 tx=0x7fb014004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb00bfff700 1 -- 172.21.15.7:0/2362402372 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01401d070 con 0x7fb024100320 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb0292fe700 1 -- 172.21.15.7:0/2362402372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02419db00 con 0x7fb024100320 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb00bfff700 1 -- 172.21.15.7:0/2362402372 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb014022470 con 0x7fb024100320 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb00bfff700 1 -- 172.21.15.7:0/2362402372 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014017650 con 0x7fb024100320 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.969+0000 7fb0292fe700 1 -- 172.21.15.7:0/2362402372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0241a53c0 con 0x7fb024100320 2024-03-23T13:18:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.970+0000 7fb0292fe700 1 -- 172.21.15.7:0/2362402372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0241016f0 con 0x7fb024100320 2024-03-23T13:18:53.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.972+0000 7fb00bfff700 1 -- 172.21.15.7:0/2362402372 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fb0140225e0 con 0x7fb024100320 2024-03-23T13:18:53.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.973+0000 7fb00bfff700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb00c0a21e0 0x7fb00c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:53.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.973+0000 7fb0227fc700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb00c0a21e0 0x7fb00c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:53.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.973+0000 7fb00bfff700 1 -- 172.21.15.7:0/2362402372 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb0140cf5b0 con 0x7fb024100320 2024-03-23T13:18:53.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.973+0000 7fb0227fc700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb00c0a21e0 0x7fb00c0a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb0240fdf20 tx=0x7fb018009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:53.976+0000 7fb00bfff700 1 -- 172.21.15.7:0/2362402372 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb0140984c0 con 0x7fb024100320 2024-03-23T13:18:54.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:53 smithi155 ceph-mon[36157]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 12 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T13:18:54.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:53 smithi007 ceph-mon[32184]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 12 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T13:18:54.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.182+0000 7fb0292fe700 1 -- 172.21.15.7:0/2362402372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb0241999d0 con 0x7fb024100320 2024-03-23T13:18:54.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.183+0000 7fb00bfff700 1 -- 172.21.15.7:0/2362402372 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb01409bb80 con 0x7fb024100320 2024-03-23T13:18:54.184 INFO:teuthology.orchestra.run.smithi007.stdout:197568495669 2024-03-23T13:18:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 -- 172.21.15.7:0/2362402372 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb00c0a21e0 msgr2=0x7fb00c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb00c0a21e0 0x7fb00c0a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb0240fdf20 tx=0x7fb018009260 comp rx=0 tx=0).stop 2024-03-23T13:18:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 -- 172.21.15.7:0/2362402372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024100320 msgr2=0x7fb024198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024100320 0x7fb024198670 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb014000c00 tx=0x7fb014004730 comp rx=0 tx=0).stop 2024-03-23T13:18:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 -- 172.21.15.7:0/2362402372 shutdown_connections 2024-03-23T13:18:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb00c0a21e0 0x7fb00c0a46a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb024108340 0x7fb02419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024100320 0x7fb024198670 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 --2- 172.21.15.7:0/2362402372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb024100ce0 0x7fb024198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 -- 172.21.15.7:0/2362402372 >> 172.21.15.7:0/2362402372 conn(0x7fb0240772d0 msgr2=0x7fb0240fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:54.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 -- 172.21.15.7:0/2362402372 shutdown_connections 2024-03-23T13:18:54.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.185+0000 7fb0292fe700 1 -- 172.21.15.7:0/2362402372 wait complete. 2024-03-23T13:18:54.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:53 smithi130 ceph-mon[36225]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 12 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.212+0000 7f0d76a8e700 1 -- 172.21.15.7:0/983961471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70102440 msgr2=0x7f0d70102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.212+0000 7f0d76a8e700 1 --2- 172.21.15.7:0/983961471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70102440 0x7f0d70102820 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009990 tx=0x7f0d64009ca0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.212+0000 7f0d76a8e700 1 -- 172.21.15.7:0/983961471 shutdown_connections 2024-03-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.212+0000 7f0d76a8e700 1 --2- 172.21.15.7:0/983961471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d70107520 0x7f0d7010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.212+0000 7f0d76a8e700 1 --2- 172.21.15.7:0/983961471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70102440 0x7f0d70102820 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.212+0000 7f0d76a8e700 1 --2- 172.21.15.7:0/983961471 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d70102df0 0x7f0d70106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.212+0000 7f0d76a8e700 1 -- 172.21.15.7:0/983961471 >> 172.21.15.7:0/983961471 conn(0x7f0d700fdcb0 msgr2=0x7f0d701000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.213+0000 7f0d76a8e700 1 -- 172.21.15.7:0/983961471 shutdown_connections 2024-03-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.213+0000 7f0d76a8e700 1 -- 172.21.15.7:0/983961471 wait complete. 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.213+0000 7f0d76a8e700 1 Processor -- start 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.213+0000 7f0d76a8e700 1 -- start start 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.213+0000 7f0d76a8e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d70102440 0x7f0d70074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.213+0000 7f0d76a8e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70102df0 0x7f0d70074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.213+0000 7f0d76a8e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d70107520 0x7f0d70074f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.213+0000 7f0d76a8e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701a6b50 con 0x7f0d70102df0 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.213+0000 7f0d76a8e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701a6cc0 con 0x7f0d70107520 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.213+0000 7f0d76a8e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701a6e80 con 0x7f0d70102440 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70102df0 0x7f0d70074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70102df0 0x7f0d70074a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58778/0 (socket says 172.21.15.7:58778) 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6ffff700 1 -- 172.21.15.7:0/1742131505 learned_addr learned my addr 172.21.15.7:0/1742131505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d7482a700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d70102440 0x7f0d70074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d7502b700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d70107520 0x7f0d70074f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6ffff700 1 -- 172.21.15.7:0/1742131505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d70102440 msgr2=0x7f0d70074500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6ffff700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d70102440 0x7f0d70074500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6ffff700 1 -- 172.21.15.7:0/1742131505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d70107520 msgr2=0x7f0d70074f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6ffff700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d70107520 0x7f0d70074f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6ffff700 1 -- 172.21.15.7:0/1742131505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d68009c40 con 0x7f0d70102df0 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6ffff700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70102df0 0x7f0d70074a40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00e970 tx=0x7f0d5c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6dffb700 1 -- 172.21.15.7:0/1742131505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c009790 con 0x7f0d70102df0 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d76a8e700 1 -- 172.21.15.7:0/1742131505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d640095f0 con 0x7f0d70102df0 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6dffb700 1 -- 172.21.15.7:0/1742131505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d5c00c7c0 con 0x7f0d70102df0 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.214+0000 7f0d6dffb700 1 -- 172.21.15.7:0/1742131505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c0055c0 con 0x7f0d70102df0 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.215+0000 7f0d76a8e700 1 -- 172.21.15.7:0/1742131505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701a7520 con 0x7f0d70102df0 2024-03-23T13:18:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.216+0000 7f0d76a8e700 1 -- 172.21.15.7:0/1742131505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7010a7e0 con 0x7f0d70102df0 2024-03-23T13:18:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.217+0000 7f0d6dffb700 1 -- 172.21.15.7:0/1742131505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f0d5c00c930 con 0x7f0d70102df0 2024-03-23T13:18:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.218+0000 7f0d6dffb700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0d600a2260 0x7f0d600a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.218+0000 7f0d6dffb700 1 -- 172.21.15.7:0/1742131505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0d5c014070 con 0x7f0d70102df0 2024-03-23T13:18:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.218+0000 7f0d7482a700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0d600a2260 0x7f0d600a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.218+0000 7f0d7482a700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0d600a2260 0x7f0d600a4720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0d640098e0 tx=0x7f0d64005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.223+0000 7f0d6dffb700 1 -- 172.21.15.7:0/1742131505 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f0d5c096de0 con 0x7f0d70102df0 2024-03-23T13:18:54.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.428+0000 7f0d76a8e700 1 -- 172.21.15.7:0/1742131505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0d700759e0 con 0x7f0d70102df0 2024-03-23T13:18:54.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.429+0000 7f0d6dffb700 1 -- 172.21.15.7:0/1742131505 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0d5c09a4a0 con 0x7f0d70102df0 2024-03-23T13:18:54.429 INFO:teuthology.orchestra.run.smithi007.stdout:163208757306 2024-03-23T13:18:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 -- 172.21.15.7:0/1742131505 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0d600a2260 msgr2=0x7f0d600a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0d600a2260 0x7f0d600a4720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0d640098e0 tx=0x7f0d64005960 comp rx=0 tx=0).stop 2024-03-23T13:18:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 -- 172.21.15.7:0/1742131505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70102df0 msgr2=0x7f0d70074a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70102df0 0x7f0d70074a40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00e970 tx=0x7f0d5c00ec80 comp rx=0 tx=0).stop 2024-03-23T13:18:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 -- 172.21.15.7:0/1742131505 shutdown_connections 2024-03-23T13:18:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f0d600a2260 0x7f0d600a4720 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d70107520 0x7f0d70074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d70102440 0x7f0d70074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 --2- 172.21.15.7:0/1742131505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70102df0 0x7f0d70074a40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 -- 172.21.15.7:0/1742131505 >> 172.21.15.7:0/1742131505 conn(0x7f0d700fdcb0 msgr2=0x7f0d700fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 -- 172.21.15.7:0/1742131505 shutdown_connections 2024-03-23T13:18:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:54.431+0000 7f0d76a8e700 1 -- 172.21.15.7:0/1742131505 wait complete. 2024-03-23T13:18:54.985 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738450 for osd.0 2024-03-23T13:18:54.985 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:55.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:54 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2362402372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T13:18:55.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:54 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1742131505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T13:18:55.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:54 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2362402372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T13:18:55.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:54 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1742131505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T13:18:55.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:54 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2362402372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T13:18:55.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:54 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1742131505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T13:18:55.358 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411400 for osd.2 2024-03-23T13:18:55.358 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:56.001 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:55 smithi007 ceph-mon[32184]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-23T13:18:56.037 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757298 got 163208757306 for osd.5 2024-03-23T13:18:56.037 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:56.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:55 smithi155 ceph-mon[36157]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-23T13:18:56.194 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:55 smithi130 ceph-mon[36225]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-23T13:18:56.449 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495669 for osd.6 2024-03-23T13:18:56.450 DEBUG:teuthology.parallel:result is None 2024-03-23T13:18:56.450 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-23T13:18:56.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-23T13:18:57.226 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:18:58.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:57 smithi155 ceph-mon[36157]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T13:18:58.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:57 smithi007 ceph-mon[32184]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T13:18:58.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:57 smithi130 ceph-mon[36225]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T13:18:58.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.930+0000 7f1108e71700 1 -- 172.21.15.7:0/1954035757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1104108230 msgr2=0x7f1104108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.930+0000 7f1108e71700 1 --2- 172.21.15.7:0/1954035757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1104108230 0x7f1104108610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f10f40099c0 tx=0x7f10f4009cd0 comp rx=0 tx=0).stop 2024-03-23T13:18:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.930+0000 7f1108e71700 1 -- 172.21.15.7:0/1954035757 shutdown_connections 2024-03-23T13:18:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.930+0000 7f1108e71700 1 --2- 172.21.15.7:0/1954035757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1104102c10 0x7f110410f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.930+0000 7f1108e71700 1 --2- 172.21.15.7:0/1954035757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1104108230 0x7f1104108610 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.930+0000 7f1108e71700 1 --2- 172.21.15.7:0/1954035757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1104102250 0x7f11041026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.930+0000 7f1108e71700 1 -- 172.21.15.7:0/1954035757 >> 172.21.15.7:0/1954035757 conn(0x7f1104079330 msgr2=0x7f1104079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.930+0000 7f1108e71700 1 -- 172.21.15.7:0/1954035757 shutdown_connections 2024-03-23T13:18:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.930+0000 7f1108e71700 1 -- 172.21.15.7:0/1954035757 wait complete. 2024-03-23T13:18:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.931+0000 7f1108e71700 1 Processor -- start 2024-03-23T13:18:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.931+0000 7f1108e71700 1 -- start start 2024-03-23T13:18:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.931+0000 7f1108e71700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1104102250 0x7f1104198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.931+0000 7f1108e71700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1104102c10 0x7f1104198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.931+0000 7f1108e71700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1104108230 0x7f110419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f1108e71700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f110419d5e0 con 0x7f1104102250 2024-03-23T13:18:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f1108e71700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f110419d750 con 0x7f1104102c10 2024-03-23T13:18:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f1108e71700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f110419d910 con 0x7f1104108230 2024-03-23T13:18:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f110259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1104102250 0x7f1104198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f110259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1104102250 0x7f1104198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52484/0 (socket says 172.21.15.7:52484) 2024-03-23T13:18:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f110259c700 1 -- 172.21.15.7:0/1791100855 learned_addr learned my addr 172.21.15.7:0/1791100855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:18:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f1102d9d700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1104108230 0x7f110419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f1101d9b700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1104102c10 0x7f1104198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f110259c700 1 -- 172.21.15.7:0/1791100855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1104108230 msgr2=0x7f110419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f110259c700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1104108230 0x7f110419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f110259c700 1 -- 172.21.15.7:0/1791100855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1104102c10 msgr2=0x7f1104198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f110259c700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1104102c10 0x7f1104198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f110259c700 1 -- 172.21.15.7:0/1791100855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f40095f0 con 0x7f1104102250 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f110259c700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1104102250 0x7f1104198700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f10f4006010 tx=0x7f10f4005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f10f37fe700 1 -- 172.21.15.7:0/1791100855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f401d070 con 0x7f1104102250 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f10f37fe700 1 -- 172.21.15.7:0/1791100855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f400bc70 con 0x7f1104102250 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.932+0000 7f1108e71700 1 -- 172.21.15.7:0/1791100855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110419db90 con 0x7f1104102250 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.933+0000 7f10f37fe700 1 -- 172.21.15.7:0/1791100855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f400fcc0 con 0x7f1104102250 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.933+0000 7f1108e71700 1 -- 172.21.15.7:0/1791100855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11041a5450 con 0x7f1104102250 2024-03-23T13:18:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.933+0000 7f1108e71700 1 -- 172.21.15.7:0/1791100855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110410a710 con 0x7f1104102250 2024-03-23T13:18:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.935+0000 7f10f37fe700 1 -- 172.21.15.7:0/1791100855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f10f400f460 con 0x7f1104102250 2024-03-23T13:18:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.936+0000 7f10f37fe700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f10ec0a21e0 0x7f10ec0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:18:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.936+0000 7f1101d9b700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f10ec0a21e0 0x7f10ec0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:18:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.936+0000 7f10f37fe700 1 -- 172.21.15.7:0/1791100855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f10f40ceea0 con 0x7f1104102250 2024-03-23T13:18:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.936+0000 7f1101d9b700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f10ec0a21e0 0x7f10ec0a46a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f10f8005fd0 tx=0x7f10f8005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:18:58.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:58.939+0000 7f10f37fe700 1 -- 172.21.15.7:0/1791100855 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f10f4097db0 con 0x7f1104102250 2024-03-23T13:18:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.166+0000 7f1108e71700 1 -- 172.21.15.7:0/1791100855 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1104199a60 con 0x7f10ec0a21e0 2024-03-23T13:18:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.173+0000 7f10f37fe700 1 -- 172.21.15.7:0/1791100855 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258968 (secure 0 0 0) 0x7f1104199a60 con 0x7f10ec0a21e0 2024-03-23T13:18:59.178 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:18:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 -- 172.21.15.7:0/1791100855 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f10ec0a21e0 msgr2=0x7f10ec0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f10ec0a21e0 0x7f10ec0a46a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f10f8005fd0 tx=0x7f10f8005f60 comp rx=0 tx=0).stop 2024-03-23T13:18:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 -- 172.21.15.7:0/1791100855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1104102250 msgr2=0x7f1104198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:18:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1104102250 0x7f1104198700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f10f4006010 tx=0x7f10f4005280 comp rx=0 tx=0).stop 2024-03-23T13:18:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 -- 172.21.15.7:0/1791100855 shutdown_connections 2024-03-23T13:18:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f10ec0a21e0 0x7f10ec0a46a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1104108230 0x7f110419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1104102250 0x7f1104198700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 --2- 172.21.15.7:0/1791100855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1104102c10 0x7f1104198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:18:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 -- 172.21.15.7:0/1791100855 >> 172.21.15.7:0/1791100855 conn(0x7f1104079330 msgr2=0x7f1104100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:18:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 -- 172.21.15.7:0/1791100855 shutdown_connections 2024-03-23T13:18:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:18:59.176+0000 7f1108e71700 1 -- 172.21.15.7:0/1791100855 wait complete. 2024-03-23T13:18:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-23T13:18:59.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:59 smithi007 ceph-mon[32184]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T13:18:59.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:59.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:18:59 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:18:59.824 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-03-23T13:18:57.942600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654367,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":852,"num_read_kb":545,"num_write":418,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373392,"kb_used_data":11688,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568432,"statfs":{"total":1151940427776,"available":1151558074368,"internally_reserved":0,"allocated":11968512,"data_stored":7425945,"data_compressed":113961,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":74,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003128"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673757+0000","last_change":"2024-03-23T13:17:41.250141+0000","last_active":"2024-03-23T13:18:43.673757+0000","last_peered":"2024-03-23T13:18:43.673757+0000","last_clean":"2024-03-23T13:18:43.673757+0000","last_became_active":"2024-03-23T13:17:41.249578+0000","last_became_peered":"2024-03-23T13:17:41.249578+0000","last_unstale":"2024-03-23T13:18:43.673757+0000","last_undegraded":"2024-03-23T13:18:43.673757+0000","last_fullsized":"2024-03-23T13:18:43.673757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:45.858553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670910+0000","last_change":"2024-03-23T13:17:41.251945+0000","last_active":"2024-03-23T13:18:43.670910+0000","last_peered":"2024-03-23T13:18:43.670910+0000","last_clean":"2024-03-23T13:18:43.670910+0000","last_became_active":"2024-03-23T13:17:41.251603+0000","last_became_peered":"2024-03-23T13:17:41.251603+0000","last_unstale":"2024-03-23T13:18:43.670910+0000","last_undegraded":"2024-03-23T13:18:43.670910+0000","last_fullsized":"2024-03-23T13:18:43.670910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:08:55.160949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670770+0000","last_change":"2024-03-23T13:17:41.254293+0000","last_active":"2024-03-23T13:18:43.670770+0000","last_peered":"2024-03-23T13:18:43.670770+0000","last_clean":"2024-03-23T13:18:43.670770+0000","last_became_active":"2024-03-23T13:17:41.253981+0000","last_became_peered":"2024-03-23T13:17:41.253981+0000","last_unstale":"2024-03-23T13:18:43.670770+0000","last_undegraded":"2024-03-23T13:18:43.670770+0000","last_fullsized":"2024-03-23T13:18:43.670770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:23:17.445169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.832344+0000","last_change":"2024-03-23T13:17:41.252125+0000","last_active":"2024-03-23T13:18:43.832344+0000","last_peered":"2024-03-23T13:18:43.832344+0000","last_clean":"2024-03-23T13:18:43.832344+0000","last_became_active":"2024-03-23T13:17:41.251819+0000","last_became_peered":"2024-03-23T13:17:41.251819+0000","last_unstale":"2024-03-23T13:18:43.832344+0000","last_undegraded":"2024-03-23T13:18:43.832344+0000","last_fullsized":"2024-03-23T13:18:43.832344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:31.559237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670723+0000","last_change":"2024-03-23T13:17:41.244653+0000","last_active":"2024-03-23T13:18:43.670723+0000","last_peered":"2024-03-23T13:18:43.670723+0000","last_clean":"2024-03-23T13:18:43.670723+0000","last_became_active":"2024-03-23T13:17:41.244322+0000","last_became_peered":"2024-03-23T13:17:41.244322+0000","last_unstale":"2024-03-23T13:18:43.670723+0000","last_undegraded":"2024-03-23T13:18:43.670723+0000","last_fullsized":"2024-03-23T13:18:43.670723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:55:19.419296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.676631+0000","last_change":"2024-03-23T13:18:43.676631+0000","last_active":"2024-03-23T13:18:43.676631+0000","last_peered":"2024-03-23T13:18:43.676631+0000","last_clean":"2024-03-23T13:18:43.676631+0000","last_became_active":"2024-03-23T13:18:43.676219+0000","last_became_peered":"2024-03-23T13:18:43.676219+0000","last_unstale":"2024-03-23T13:18:43.676631+0000","last_undegraded":"2024-03-23T13:18:43.676631+0000","last_fullsized":"2024-03-23T13:18:43.676631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:42:20.673553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670785+0000","last_change":"2024-03-23T13:17:41.250721+0000","last_active":"2024-03-23T13:18:43.670785+0000","last_peered":"2024-03-23T13:18:43.670785+0000","last_clean":"2024-03-23T13:18:43.670785+0000","last_became_active":"2024-03-23T13:17:41.250403+0000","last_became_peered":"2024-03-23T13:17:41.250403+0000","last_unstale":"2024-03-23T13:18:43.670785+0000","last_undegraded":"2024-03-23T13:18:43.670785+0000","last_fullsized":"2024-03-23T13:18:43.670785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:51:54.657066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.088565+0000","last_change":"2024-03-23T13:18:44.088565+0000","last_active":"2024-03-23T13:18:44.088565+0000","last_peered":"2024-03-23T13:18:44.088565+0000","last_clean":"2024-03-23T13:18:44.088565+0000","last_became_active":"2024-03-23T13:18:44.088216+0000","last_became_peered":"2024-03-23T13:18:44.088216+0000","last_unstale":"2024-03-23T13:18:44.088565+0000","last_undegraded":"2024-03-23T13:18:44.088565+0000","last_fullsized":"2024-03-23T13:18:44.088565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:30:40.102611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.672771+0000","last_change":"2024-03-23T13:17:41.253829+0000","last_active":"2024-03-23T13:18:43.672771+0000","last_peered":"2024-03-23T13:18:43.672771+0000","last_clean":"2024-03-23T13:18:43.672771+0000","last_became_active":"2024-03-23T13:17:41.253520+0000","last_became_peered":"2024-03-23T13:17:41.253520+0000","last_unstale":"2024-03-23T13:18:43.672771+0000","last_undegraded":"2024-03-23T13:18:43.672771+0000","last_fullsized":"2024-03-23T13:18:43.672771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:27:40.159325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.679268+0000","last_change":"2024-03-23T13:18:43.679268+0000","last_active":"2024-03-23T13:18:43.679268+0000","last_peered":"2024-03-23T13:18:43.679268+0000","last_clean":"2024-03-23T13:18:43.679268+0000","last_became_active":"2024-03-23T13:18:43.678825+0000","last_became_peered":"2024-03-23T13:18:43.678825+0000","last_unstale":"2024-03-23T13:18:43.679268+0000","last_undegraded":"2024-03-23T13:18:43.679268+0000","last_fullsized":"2024-03-23T13:18:43.679268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:55:22.999752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671435+0000","last_change":"2024-03-23T13:17:41.257957+0000","last_active":"2024-03-23T13:18:43.671435+0000","last_peered":"2024-03-23T13:18:43.671435+0000","last_clean":"2024-03-23T13:18:43.671435+0000","last_became_active":"2024-03-23T13:17:41.257651+0000","last_became_peered":"2024-03-23T13:17:41.257651+0000","last_unstale":"2024-03-23T13:18:43.671435+0000","last_undegraded":"2024-03-23T13:18:43.671435+0000","last_fullsized":"2024-03-23T13:18:43.671435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:42:33.120763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.832332+0000","last_change":"2024-03-23T13:17:41.254935+0000","last_active":"2024-03-23T13:18:43.832332+0000","last_peered":"2024-03-23T13:18:43.832332+0000","last_clean":"2024-03-23T13:18:43.832332+0000","last_became_active":"2024-03-23T13:17:41.254633+0000","last_became_peered":"2024-03-23T13:17:41.254633+0000","last_unstale":"2024-03-23T13:18:43.832332+0000","last_undegraded":"2024-03-23T13:18:43.832332+0000","last_fullsized":"2024-03-23T13:18:43.832332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:23:18.345884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183385+0000","last_change":"2024-03-23T13:17:41.233447+0000","last_active":"2024-03-23T13:18:44.183385+0000","last_peered":"2024-03-23T13:18:44.183385+0000","last_clean":"2024-03-23T13:18:44.183385+0000","last_became_active":"2024-03-23T13:17:41.233143+0000","last_became_peered":"2024-03-23T13:17:41.233143+0000","last_unstale":"2024-03-23T13:18:44.183385+0000","last_undegraded":"2024-03-23T13:18:44.183385+0000","last_fullsized":"2024-03-23T13:18:44.183385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:16:53.677024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670101+0000","last_change":"2024-03-23T13:17:41.242668+0000","last_active":"2024-03-23T13:18:43.670101+0000","last_peered":"2024-03-23T13:18:43.670101+0000","last_clean":"2024-03-23T13:18:43.670101+0000","last_became_active":"2024-03-23T13:17:41.242321+0000","last_became_peered":"2024-03-23T13:17:41.242321+0000","last_unstale":"2024-03-23T13:18:43.670101+0000","last_undegraded":"2024-03-23T13:18:43.670101+0000","last_fullsized":"2024-03-23T13:18:43.670101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:17.783814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.184349+0000","last_change":"2024-03-23T13:18:44.184349+0000","last_active":"2024-03-23T13:18:44.184349+0000","last_peered":"2024-03-23T13:18:44.184349+0000","last_clean":"2024-03-23T13:18:44.184349+0000","last_became_active":"2024-03-23T13:18:44.183981+0000","last_became_peered":"2024-03-23T13:18:44.183981+0000","last_unstale":"2024-03-23T13:18:44.184349+0000","last_undegraded":"2024-03-23T13:18:44.184349+0000","last_fullsized":"2024-03-23T13:18:44.184349+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:45:08.419580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671717+0000","last_change":"2024-03-23T13:17:41.253679+0000","last_active":"2024-03-23T13:18:43.671717+0000","last_peered":"2024-03-23T13:18:43.671717+0000","last_clean":"2024-03-23T13:18:43.671717+0000","last_became_active":"2024-03-23T13:17:41.253371+0000","last_became_peered":"2024-03-23T13:17:41.253371+0000","last_unstale":"2024-03-23T13:18:43.671717+0000","last_undegraded":"2024-03-23T13:18:43.671717+0000","last_fullsized":"2024-03-23T13:18:43.671717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:16:08.397446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673853+0000","last_change":"2024-03-23T13:17:41.249856+0000","last_active":"2024-03-23T13:18:43.673853+0000","last_peered":"2024-03-23T13:18:43.673853+0000","last_clean":"2024-03-23T13:18:43.673853+0000","last_became_active":"2024-03-23T13:17:41.249163+0000","last_became_peered":"2024-03-23T13:17:41.249163+0000","last_unstale":"2024-03-23T13:18:43.673853+0000","last_undegraded":"2024-03-23T13:18:43.673853+0000","last_fullsized":"2024-03-23T13:18:43.673853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:14.184859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673543+0000","last_change":"2024-03-23T13:17:41.250048+0000","last_active":"2024-03-23T13:18:43.673543+0000","last_peered":"2024-03-23T13:18:43.673543+0000","last_clean":"2024-03-23T13:18:43.673543+0000","last_became_active":"2024-03-23T13:17:41.249491+0000","last_became_peered":"2024-03-23T13:17:41.249491+0000","last_unstale":"2024-03-23T13:18:43.673543+0000","last_undegraded":"2024-03-23T13:18:43.673543+0000","last_fullsized":"2024-03-23T13:18:43.673543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:55:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.675991+0000","last_change":"2024-03-23T13:17:41.256615+0000","last_active":"2024-03-23T13:18:43.675991+0000","last_peered":"2024-03-23T13:18:43.675991+0000","last_clean":"2024-03-23T13:18:43.675991+0000","last_became_active":"2024-03-23T13:17:41.256287+0000","last_became_peered":"2024-03-23T13:17:41.256287+0000","last_unstale":"2024-03-23T13:18:43.675991+0000","last_undegraded":"2024-03-23T13:18:43.675991+0000","last_fullsized":"2024-03-23T13:18:43.675991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:31.686699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:56.346078+0000","last_change":"2024-03-23T13:17:40.231464+0000","last_active":"2024-03-23T13:18:56.346078+0000","last_peered":"2024-03-23T13:18:56.346078+0000","last_clean":"2024-03-23T13:18:56.346078+0000","last_became_active":"2024-03-23T13:17:40.230711+0000","last_became_peered":"2024-03-23T13:17:40.230711+0000","last_unstale":"2024-03-23T13:18:56.346078+0000","last_undegraded":"2024-03-23T13:18:56.346078+0000","last_fullsized":"2024-03-23T13:18:56.346078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:17.926187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879850+0000","last_change":"2024-03-23T13:17:41.251449+0000","last_active":"2024-03-23T13:18:44.879850+0000","last_peered":"2024-03-23T13:18:44.879850+0000","last_clean":"2024-03-23T13:18:44.879850+0000","last_became_active":"2024-03-23T13:17:41.251077+0000","last_became_peered":"2024-03-23T13:17:41.251077+0000","last_unstale":"2024-03-23T13:18:44.879850+0000","last_undegraded":"2024-03-23T13:18:44.879850+0000","last_fullsized":"2024-03-23T13:18:44.879850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:09.939244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879189+0000","last_change":"2024-03-23T13:17:40.233263+0000","last_active":"2024-03-23T13:18:44.879189+0000","last_peered":"2024-03-23T13:18:44.879189+0000","last_clean":"2024-03-23T13:18:44.879189+0000","last_became_active":"2024-03-23T13:17:40.232686+0000","last_became_peered":"2024-03-23T13:17:40.232686+0000","last_unstale":"2024-03-23T13:18:44.879189+0000","last_undegraded":"2024-03-23T13:18:44.879189+0000","last_fullsized":"2024-03-23T13:18:44.879189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:26.671088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671009+0000","last_change":"2024-03-23T13:17:41.250532+0000","last_active":"2024-03-23T13:18:43.671009+0000","last_peered":"2024-03-23T13:18:43.671009+0000","last_clean":"2024-03-23T13:18:43.671009+0000","last_became_active":"2024-03-23T13:17:41.250230+0000","last_became_peered":"2024-03-23T13:17:41.250230+0000","last_unstale":"2024-03-23T13:18:43.671009+0000","last_undegraded":"2024-03-23T13:18:43.671009+0000","last_fullsized":"2024-03-23T13:18:43.671009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:13:27.406827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.843761+0000","last_change":"2024-03-23T13:18:43.843761+0000","last_active":"2024-03-23T13:18:43.843761+0000","last_peered":"2024-03-23T13:18:43.843761+0000","last_clean":"2024-03-23T13:18:43.843761+0000","last_became_active":"2024-03-23T13:18:43.833436+0000","last_became_peered":"2024-03-23T13:18:43.833436+0000","last_unstale":"2024-03-23T13:18:43.843761+0000","last_undegraded":"2024-03-23T13:18:43.843761+0000","last_fullsized":"2024-03-23T13:18:43.843761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:49:01.749814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.672057+0000","last_change":"2024-03-23T13:17:41.247529+0000","last_active":"2024-03-23T13:18:43.672057+0000","last_peered":"2024-03-23T13:18:43.672057+0000","last_clean":"2024-03-23T13:18:43.672057+0000","last_became_active":"2024-03-23T13:17:41.247008+0000","last_became_peered":"2024-03-23T13:17:41.247008+0000","last_unstale":"2024-03-23T13:18:43.672057+0000","last_undegraded":"2024-03-23T13:18:43.672057+0000","last_fullsized":"2024-03-23T13:18:43.672057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:22.015602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671864+0000","last_change":"2024-03-23T13:17:40.233544+0000","last_active":"2024-03-23T13:18:43.671864+0000","last_peered":"2024-03-23T13:18:43.671864+0000","last_clean":"2024-03-23T13:18:43.671864+0000","last_became_active":"2024-03-23T13:17:40.233222+0000","last_became_peered":"2024-03-23T13:17:40.233222+0000","last_unstale":"2024-03-23T13:18:43.671864+0000","last_undegraded":"2024-03-23T13:18:43.671864+0000","last_fullsized":"2024-03-23T13:18:43.671864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:15:32.147020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087543+0000","last_change":"2024-03-23T13:17:41.245643+0000","last_active":"2024-03-23T13:18:44.087543+0000","last_peered":"2024-03-23T13:18:44.087543+0000","last_clean":"2024-03-23T13:18:44.087543+0000","last_became_active":"2024-03-23T13:17:41.245295+0000","last_became_peered":"2024-03-23T13:17:41.245295+0000","last_unstale":"2024-03-23T13:18:44.087543+0000","last_undegraded":"2024-03-23T13:18:44.087543+0000","last_fullsized":"2024-03-23T13:18:44.087543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:27:26.480225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:54.337564+0000","last_change":"2024-03-23T13:17:40.236473+0000","last_active":"2024-03-23T13:18:54.337564+0000","last_peered":"2024-03-23T13:18:54.337564+0000","last_clean":"2024-03-23T13:18:54.337564+0000","last_became_active":"2024-03-23T13:17:40.236114+0000","last_became_peered":"2024-03-23T13:17:40.236114+0000","last_unstale":"2024-03-23T13:18:54.337564+0000","last_undegraded":"2024-03-23T13:18:54.337564+0000","last_fullsized":"2024-03-23T13:18:54.337564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:37:33.949014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670614+0000","last_change":"2024-03-23T13:17:41.241904+0000","last_active":"2024-03-23T13:18:43.670614+0000","last_peered":"2024-03-23T13:18:43.670614+0000","last_clean":"2024-03-23T13:18:43.670614+0000","last_became_active":"2024-03-23T13:17:41.241605+0000","last_became_peered":"2024-03-23T13:17:41.241605+0000","last_unstale":"2024-03-23T13:18:43.670614+0000","last_undegraded":"2024-03-23T13:18:43.670614+0000","last_fullsized":"2024-03-23T13:18:43.670614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:05:43.782714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674124+0000","last_change":"2024-03-23T13:17:40.232399+0000","last_active":"2024-03-23T13:18:43.674124+0000","last_peered":"2024-03-23T13:18:43.674124+0000","last_clean":"2024-03-23T13:18:43.674124+0000","last_became_active":"2024-03-23T13:17:40.232071+0000","last_became_peered":"2024-03-23T13:17:40.232071+0000","last_unstale":"2024-03-23T13:18:43.674124+0000","last_undegraded":"2024-03-23T13:18:43.674124+0000","last_fullsized":"2024-03-23T13:18:43.674124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:34:35.839756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087321+0000","last_change":"2024-03-23T13:17:41.246698+0000","last_active":"2024-03-23T13:18:44.087321+0000","last_peered":"2024-03-23T13:18:44.087321+0000","last_clean":"2024-03-23T13:18:44.087321+0000","last_became_active":"2024-03-23T13:17:41.246351+0000","last_became_peered":"2024-03-23T13:17:41.246351+0000","last_unstale":"2024-03-23T13:18:44.087321+0000","last_undegraded":"2024-03-23T13:18:44.087321+0000","last_fullsized":"2024-03-23T13:18:44.087321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:35:43.068833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183499+0000","last_change":"2024-03-23T13:17:40.230550+0000","last_active":"2024-03-23T13:18:44.183499+0000","last_peered":"2024-03-23T13:18:44.183499+0000","last_clean":"2024-03-23T13:18:44.183499+0000","last_became_active":"2024-03-23T13:17:40.230215+0000","last_became_peered":"2024-03-23T13:17:40.230215+0000","last_unstale":"2024-03-23T13:18:44.183499+0000","last_undegraded":"2024-03-23T13:18:44.183499+0000","last_fullsized":"2024-03-23T13:18:44.183499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:48:52.998612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879253+0000","last_change":"2024-03-23T13:17:41.246704+0000","last_active":"2024-03-23T13:18:44.879253+0000","last_peered":"2024-03-23T13:18:44.879253+0000","last_clean":"2024-03-23T13:18:44.879253+0000","last_became_active":"2024-03-23T13:17:41.246405+0000","last_became_peered":"2024-03-23T13:17:41.246405+0000","last_unstale":"2024-03-23T13:18:44.879253+0000","last_undegraded":"2024-03-23T13:18:44.879253+0000","last_fullsized":"2024-03-23T13:18:44.879253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:23:10.404830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087283+0000","last_change":"2024-03-23T13:17:40.232444+0000","last_active":"2024-03-23T13:18:44.087283+0000","last_peered":"2024-03-23T13:18:44.087283+0000","last_clean":"2024-03-23T13:18:44.087283+0000","last_became_active":"2024-03-23T13:17:40.232142+0000","last_became_peered":"2024-03-23T13:17:40.232142+0000","last_unstale":"2024-03-23T13:18:44.087283+0000","last_undegraded":"2024-03-23T13:18:44.087283+0000","last_fullsized":"2024-03-23T13:18:44.087283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:32:58.171626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673241+0000","last_change":"2024-03-23T13:17:41.246894+0000","last_active":"2024-03-23T13:18:43.673241+0000","last_peered":"2024-03-23T13:18:43.673241+0000","last_clean":"2024-03-23T13:18:43.673241+0000","last_became_active":"2024-03-23T13:17:41.246537+0000","last_became_peered":"2024-03-23T13:17:41.246537+0000","last_unstale":"2024-03-23T13:18:43.673241+0000","last_undegraded":"2024-03-23T13:18:43.673241+0000","last_fullsized":"2024-03-23T13:18:43.673241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:58.878823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":163,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:57.339930+0000","last_change":"2024-03-23T13:17:40.231214+0000","last_active":"2024-03-23T13:18:57.339930+0000","last_peered":"2024-03-23T13:18:57.339930+0000","last_clean":"2024-03-23T13:18:57.339930+0000","last_became_active":"2024-03-23T13:17:40.230890+0000","last_became_peered":"2024-03-23T13:17:40.230890+0000","last_unstale":"2024-03-23T13:18:57.339930+0000","last_undegraded":"2024-03-23T13:18:57.339930+0000","last_fullsized":"2024-03-23T13:18:57.339930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:00:57.300516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.672582+0000","last_change":"2024-03-23T13:17:41.245560+0000","last_active":"2024-03-23T13:18:43.672582+0000","last_peered":"2024-03-23T13:18:43.672582+0000","last_clean":"2024-03-23T13:18:43.672582+0000","last_became_active":"2024-03-23T13:17:41.245192+0000","last_became_peered":"2024-03-23T13:17:41.245192+0000","last_unstale":"2024-03-23T13:18:43.672582+0000","last_undegraded":"2024-03-23T13:18:43.672582+0000","last_fullsized":"2024-03-23T13:18:43.672582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:07:02.080036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670535+0000","last_change":"2024-03-23T13:17:40.232603+0000","last_active":"2024-03-23T13:18:43.670535+0000","last_peered":"2024-03-23T13:18:43.670535+0000","last_clean":"2024-03-23T13:18:43.670535+0000","last_became_active":"2024-03-23T13:17:40.231970+0000","last_became_peered":"2024-03-23T13:17:40.231970+0000","last_unstale":"2024-03-23T13:18:43.670535+0000","last_undegraded":"2024-03-23T13:18:43.670535+0000","last_fullsized":"2024-03-23T13:18:43.670535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:14:22.383047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671001+0000","last_change":"2024-03-23T13:17:41.244255+0000","last_active":"2024-03-23T13:18:43.671001+0000","last_peered":"2024-03-23T13:18:43.671001+0000","last_clean":"2024-03-23T13:18:43.671001+0000","last_became_active":"2024-03-23T13:17:41.243925+0000","last_became_peered":"2024-03-23T13:17:41.243925+0000","last_unstale":"2024-03-23T13:18:43.671001+0000","last_undegraded":"2024-03-23T13:18:43.671001+0000","last_fullsized":"2024-03-23T13:18:43.671001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:03.115703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087570+0000","last_change":"2024-03-23T13:17:40.229160+0000","last_active":"2024-03-23T13:18:44.087570+0000","last_peered":"2024-03-23T13:18:44.087570+0000","last_clean":"2024-03-23T13:18:44.087570+0000","last_became_active":"2024-03-23T13:17:40.228846+0000","last_became_peered":"2024-03-23T13:17:40.228846+0000","last_unstale":"2024-03-23T13:18:44.087570+0000","last_undegraded":"2024-03-23T13:18:44.087570+0000","last_fullsized":"2024-03-23T13:18:44.087570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:25.517124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.672225+0000","last_change":"2024-03-23T13:17:41.247364+0000","last_active":"2024-03-23T13:18:43.672225+0000","last_peered":"2024-03-23T13:18:43.672225+0000","last_clean":"2024-03-23T13:18:43.672225+0000","last_became_active":"2024-03-23T13:17:41.246671+0000","last_became_peered":"2024-03-23T13:17:41.246671+0000","last_unstale":"2024-03-23T13:18:43.672225+0000","last_undegraded":"2024-03-23T13:18:43.672225+0000","last_fullsized":"2024-03-23T13:18:43.672225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:50:15.654396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'28","reported_seq":194,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:57.339796+0000","last_change":"2024-03-23T13:17:40.229805+0000","last_active":"2024-03-23T13:18:57.339796+0000","last_peered":"2024-03-23T13:18:57.339796+0000","last_clean":"2024-03-23T13:18:57.339796+0000","last_became_active":"2024-03-23T13:17:40.229438+0000","last_became_peered":"2024-03-23T13:17:40.229438+0000","last_unstale":"2024-03-23T13:18:57.339796+0000","last_undegraded":"2024-03-23T13:18:57.339796+0000","last_fullsized":"2024-03-23T13:18:57.339796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:21:11.121141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56561,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":30,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670810+0000","last_change":"2024-03-23T13:17:41.250106+0000","last_active":"2024-03-23T13:18:43.670810+0000","last_peered":"2024-03-23T13:18:43.670810+0000","last_clean":"2024-03-23T13:18:43.670810+0000","last_became_active":"2024-03-23T13:17:41.249725+0000","last_became_peered":"2024-03-23T13:17:41.249725+0000","last_unstale":"2024-03-23T13:18:43.670810+0000","last_undegraded":"2024-03-23T13:18:43.670810+0000","last_fullsized":"2024-03-23T13:18:43.670810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:07:20.243310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.675154+0000","last_change":"2024-03-23T13:17:40.234631+0000","last_active":"2024-03-23T13:18:43.675154+0000","last_peered":"2024-03-23T13:18:43.675154+0000","last_clean":"2024-03-23T13:18:43.675154+0000","last_became_active":"2024-03-23T13:17:40.234294+0000","last_became_peered":"2024-03-23T13:17:40.234294+0000","last_unstale":"2024-03-23T13:18:43.675154+0000","last_undegraded":"2024-03-23T13:18:43.675154+0000","last_fullsized":"2024-03-23T13:18:43.675154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:57.058252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183269+0000","last_change":"2024-03-23T13:17:41.247425+0000","last_active":"2024-03-23T13:18:44.183269+0000","last_peered":"2024-03-23T13:18:44.183269+0000","last_clean":"2024-03-23T13:18:44.183269+0000","last_became_active":"2024-03-23T13:17:41.247125+0000","last_became_peered":"2024-03-23T13:17:41.247125+0000","last_unstale":"2024-03-23T13:18:44.183269+0000","last_undegraded":"2024-03-23T13:18:44.183269+0000","last_fullsized":"2024-03-23T13:18:44.183269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:50.118666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087147+0000","last_change":"2024-03-23T13:17:40.236179+0000","last_active":"2024-03-23T13:18:44.087147+0000","last_peered":"2024-03-23T13:18:44.087147+0000","last_clean":"2024-03-23T13:18:44.087147+0000","last_became_active":"2024-03-23T13:17:40.235810+0000","last_became_peered":"2024-03-23T13:17:40.235810+0000","last_unstale":"2024-03-23T13:18:44.087147+0000","last_undegraded":"2024-03-23T13:18:44.087147+0000","last_fullsized":"2024-03-23T13:18:44.087147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:27:28.193164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671081+0000","last_change":"2024-03-23T13:17:41.252261+0000","last_active":"2024-03-23T13:18:43.671081+0000","last_peered":"2024-03-23T13:18:43.671081+0000","last_clean":"2024-03-23T13:18:43.671081+0000","last_became_active":"2024-03-23T13:17:41.251949+0000","last_became_peered":"2024-03-23T13:17:41.251949+0000","last_unstale":"2024-03-23T13:18:43.671081+0000","last_undegraded":"2024-03-23T13:18:43.671081+0000","last_fullsized":"2024-03-23T13:18:43.671081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:39.576999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:54.344058+0000","last_change":"2024-03-23T13:17:40.228738+0000","last_active":"2024-03-23T13:18:54.344058+0000","last_peered":"2024-03-23T13:18:54.344058+0000","last_clean":"2024-03-23T13:18:54.344058+0000","last_became_active":"2024-03-23T13:17:40.228424+0000","last_became_peered":"2024-03-23T13:17:40.228424+0000","last_unstale":"2024-03-23T13:18:54.344058+0000","last_undegraded":"2024-03-23T13:18:54.344058+0000","last_fullsized":"2024-03-23T13:18:54.344058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:51.506223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670265+0000","last_change":"2024-03-23T13:17:41.247155+0000","last_active":"2024-03-23T13:18:43.670265+0000","last_peered":"2024-03-23T13:18:43.670265+0000","last_clean":"2024-03-23T13:18:43.670265+0000","last_became_active":"2024-03-23T13:17:41.246849+0000","last_became_peered":"2024-03-23T13:17:41.246849+0000","last_unstale":"2024-03-23T13:18:43.670265+0000","last_undegraded":"2024-03-23T13:18:43.670265+0000","last_fullsized":"2024-03-23T13:18:43.670265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:23.311456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670433+0000","last_change":"2024-03-23T13:17:40.232420+0000","last_active":"2024-03-23T13:18:43.670433+0000","last_peered":"2024-03-23T13:18:43.670433+0000","last_clean":"2024-03-23T13:18:43.670433+0000","last_became_active":"2024-03-23T13:17:40.231940+0000","last_became_peered":"2024-03-23T13:17:40.231940+0000","last_unstale":"2024-03-23T13:18:43.670433+0000","last_undegraded":"2024-03-23T13:18:43.670433+0000","last_fullsized":"2024-03-23T13:18:43.670433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:46.300579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.086994+0000","last_change":"2024-03-23T13:17:41.249717+0000","last_active":"2024-03-23T13:18:44.086994+0000","last_peered":"2024-03-23T13:18:44.086994+0000","last_clean":"2024-03-23T13:18:44.086994+0000","last_became_active":"2024-03-23T13:17:41.249286+0000","last_became_peered":"2024-03-23T13:17:41.249286+0000","last_unstale":"2024-03-23T13:18:44.086994+0000","last_undegraded":"2024-03-23T13:18:44.086994+0000","last_fullsized":"2024-03-23T13:18:44.086994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:47:19.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670921+0000","last_change":"2024-03-23T13:17:40.229172+0000","last_active":"2024-03-23T13:18:43.670921+0000","last_peered":"2024-03-23T13:18:43.670921+0000","last_clean":"2024-03-23T13:18:43.670921+0000","last_became_active":"2024-03-23T13:17:40.228858+0000","last_became_peered":"2024-03-23T13:17:40.228858+0000","last_unstale":"2024-03-23T13:18:43.670921+0000","last_undegraded":"2024-03-23T13:18:43.670921+0000","last_fullsized":"2024-03-23T13:18:43.670921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:17.929936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673485+0000","last_change":"2024-03-23T13:17:41.253041+0000","last_active":"2024-03-23T13:18:43.673485+0000","last_peered":"2024-03-23T13:18:43.673485+0000","last_clean":"2024-03-23T13:18:43.673485+0000","last_became_active":"2024-03-23T13:17:41.252739+0000","last_became_peered":"2024-03-23T13:17:41.252739+0000","last_unstale":"2024-03-23T13:18:43.673485+0000","last_undegraded":"2024-03-23T13:18:43.673485+0000","last_fullsized":"2024-03-23T13:18:43.673485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:27:52.560688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673447+0000","last_change":"2024-03-23T13:17:40.228098+0000","last_active":"2024-03-23T13:18:43.673447+0000","last_peered":"2024-03-23T13:18:43.673447+0000","last_clean":"2024-03-23T13:18:43.673447+0000","last_became_active":"2024-03-23T13:17:40.227778+0000","last_became_peered":"2024-03-23T13:17:40.227778+0000","last_unstale":"2024-03-23T13:18:43.673447+0000","last_undegraded":"2024-03-23T13:18:43.673447+0000","last_fullsized":"2024-03-23T13:18:43.673447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:25:38.862329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674553+0000","last_change":"2024-03-23T13:17:41.242666+0000","last_active":"2024-03-23T13:18:43.674553+0000","last_peered":"2024-03-23T13:18:43.674553+0000","last_clean":"2024-03-23T13:18:43.674553+0000","last_became_active":"2024-03-23T13:17:41.242371+0000","last_became_peered":"2024-03-23T13:17:41.242371+0000","last_unstale":"2024-03-23T13:18:43.674553+0000","last_undegraded":"2024-03-23T13:18:43.674553+0000","last_fullsized":"2024-03-23T13:18:43.674553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:40.870090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:53.343197+0000","last_change":"2024-03-23T13:17:40.233558+0000","last_active":"2024-03-23T13:18:53.343197+0000","last_peered":"2024-03-23T13:18:53.343197+0000","last_clean":"2024-03-23T13:18:53.343197+0000","last_became_active":"2024-03-23T13:17:40.232112+0000","last_became_peered":"2024-03-23T13:17:40.232112+0000","last_unstale":"2024-03-23T13:18:53.343197+0000","last_undegraded":"2024-03-23T13:18:53.343197+0000","last_fullsized":"2024-03-23T13:18:53.343197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:37:43.971598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674339+0000","last_change":"2024-03-23T13:17:41.245804+0000","last_active":"2024-03-23T13:18:43.674339+0000","last_peered":"2024-03-23T13:18:43.674339+0000","last_clean":"2024-03-23T13:18:43.674339+0000","last_became_active":"2024-03-23T13:17:41.245509+0000","last_became_peered":"2024-03-23T13:17:41.245509+0000","last_unstale":"2024-03-23T13:18:43.674339+0000","last_undegraded":"2024-03-23T13:18:43.674339+0000","last_fullsized":"2024-03-23T13:18:43.674339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:39:55.735413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087382+0000","last_change":"2024-03-23T13:17:41.250553+0000","last_active":"2024-03-23T13:18:44.087382+0000","last_peered":"2024-03-23T13:18:44.087382+0000","last_clean":"2024-03-23T13:18:44.087382+0000","last_became_active":"2024-03-23T13:17:41.250140+0000","last_became_peered":"2024-03-23T13:17:41.250140+0000","last_unstale":"2024-03-23T13:18:44.087382+0000","last_undegraded":"2024-03-23T13:18:44.087382+0000","last_fullsized":"2024-03-23T13:18:44.087382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:28.088801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:54.343594+0000","last_change":"2024-03-23T13:17:40.239651+0000","last_active":"2024-03-23T13:18:54.343594+0000","last_peered":"2024-03-23T13:18:54.343594+0000","last_clean":"2024-03-23T13:18:54.343594+0000","last_became_active":"2024-03-23T13:17:40.239326+0000","last_became_peered":"2024-03-23T13:17:40.239326+0000","last_unstale":"2024-03-23T13:18:54.343594+0000","last_undegraded":"2024-03-23T13:18:54.343594+0000","last_fullsized":"2024-03-23T13:18:54.343594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:56:36.147882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879605+0000","last_change":"2024-03-23T13:17:41.253148+0000","last_active":"2024-03-23T13:18:44.879605+0000","last_peered":"2024-03-23T13:18:44.879605+0000","last_clean":"2024-03-23T13:18:44.879605+0000","last_became_active":"2024-03-23T13:17:41.252571+0000","last_became_peered":"2024-03-23T13:17:41.252571+0000","last_unstale":"2024-03-23T13:18:44.879605+0000","last_undegraded":"2024-03-23T13:18:44.879605+0000","last_fullsized":"2024-03-23T13:18:44.879605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:19:36.477844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.832199+0000","last_change":"2024-03-23T13:17:40.227457+0000","last_active":"2024-03-23T13:18:43.832199+0000","last_peered":"2024-03-23T13:18:43.832199+0000","last_clean":"2024-03-23T13:18:43.832199+0000","last_became_active":"2024-03-23T13:17:40.227149+0000","last_became_peered":"2024-03-23T13:17:40.227149+0000","last_unstale":"2024-03-23T13:18:43.832199+0000","last_undegraded":"2024-03-23T13:18:43.832199+0000","last_fullsized":"2024-03-23T13:18:43.832199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:33:20.606023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670329+0000","last_change":"2024-03-23T13:16:31.257961+0000","last_active":"2024-03-23T13:18:43.670329+0000","last_peered":"2024-03-23T13:18:43.670329+0000","last_clean":"2024-03-23T13:18:43.670329+0000","last_became_active":"2024-03-23T13:16:31.257444+0000","last_became_peered":"2024-03-23T13:16:31.257444+0000","last_unstale":"2024-03-23T13:18:43.670329+0000","last_undegraded":"2024-03-23T13:18:43.670329+0000","last_fullsized":"2024-03-23T13:18:43.670329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:13:08.847194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:13:08.847194+0000","last_clean_scrub_stamp":"2024-03-23T13:13:08.847194+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:35:07.006029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183325+0000","last_change":"2024-03-23T13:17:41.249443+0000","last_active":"2024-03-23T13:18:44.183325+0000","last_peered":"2024-03-23T13:18:44.183325+0000","last_clean":"2024-03-23T13:18:44.183325+0000","last_became_active":"2024-03-23T13:17:41.248960+0000","last_became_peered":"2024-03-23T13:17:41.248960+0000","last_unstale":"2024-03-23T13:18:44.183325+0000","last_undegraded":"2024-03-23T13:18:44.183325+0000","last_fullsized":"2024-03-23T13:18:44.183325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:35:36.819271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670430+0000","last_change":"2024-03-23T13:17:40.230268+0000","last_active":"2024-03-23T13:18:43.670430+0000","last_peered":"2024-03-23T13:18:43.670430+0000","last_clean":"2024-03-23T13:18:43.670430+0000","last_became_active":"2024-03-23T13:17:40.229955+0000","last_became_peered":"2024-03-23T13:17:40.229955+0000","last_unstale":"2024-03-23T13:18:43.670430+0000","last_undegraded":"2024-03-23T13:18:43.670430+0000","last_fullsized":"2024-03-23T13:18:43.670430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:38.977844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.685224+0000","last_change":"2024-03-23T13:18:43.684867+0000","last_active":"2024-03-23T13:18:43.685224+0000","last_peered":"2024-03-23T13:18:43.685224+0000","last_clean":"2024-03-23T13:18:43.685224+0000","last_became_active":"2024-03-23T13:18:43.676582+0000","last_became_peered":"2024-03-23T13:18:43.676582+0000","last_unstale":"2024-03-23T13:18:43.685224+0000","last_undegraded":"2024-03-23T13:18:43.685224+0000","last_fullsized":"2024-03-23T13:18:43.685224+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:31:12.274326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183400+0000","last_change":"2024-03-23T13:17:41.248807+0000","last_active":"2024-03-23T13:18:44.183400+0000","last_peered":"2024-03-23T13:18:44.183400+0000","last_clean":"2024-03-23T13:18:44.183400+0000","last_became_active":"2024-03-23T13:17:41.248401+0000","last_became_peered":"2024-03-23T13:17:41.248401+0000","last_unstale":"2024-03-23T13:18:44.183400+0000","last_undegraded":"2024-03-23T13:18:44.183400+0000","last_fullsized":"2024-03-23T13:18:44.183400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:13:30.533996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673590+0000","last_change":"2024-03-23T13:17:40.233323+0000","last_active":"2024-03-23T13:18:43.673590+0000","last_peered":"2024-03-23T13:18:43.673590+0000","last_clean":"2024-03-23T13:18:43.673590+0000","last_became_active":"2024-03-23T13:17:40.233031+0000","last_became_peered":"2024-03-23T13:17:40.233031+0000","last_unstale":"2024-03-23T13:18:43.673590+0000","last_undegraded":"2024-03-23T13:18:43.673590+0000","last_fullsized":"2024-03-23T13:18:43.673590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:55:35.870633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879908+0000","last_change":"2024-03-23T13:17:41.243662+0000","last_active":"2024-03-23T13:18:44.879908+0000","last_peered":"2024-03-23T13:18:44.879908+0000","last_clean":"2024-03-23T13:18:44.879908+0000","last_became_active":"2024-03-23T13:17:41.243310+0000","last_became_peered":"2024-03-23T13:17:41.243310+0000","last_unstale":"2024-03-23T13:18:44.879908+0000","last_undegraded":"2024-03-23T13:18:44.879908+0000","last_fullsized":"2024-03-23T13:18:44.879908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:25:06.277185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.832446+0000","last_change":"2024-03-23T13:17:40.226698+0000","last_active":"2024-03-23T13:18:43.832446+0000","last_peered":"2024-03-23T13:18:43.832446+0000","last_clean":"2024-03-23T13:18:43.832446+0000","last_became_active":"2024-03-23T13:17:40.226305+0000","last_became_peered":"2024-03-23T13:17:40.226305+0000","last_unstale":"2024-03-23T13:18:43.832446+0000","last_undegraded":"2024-03-23T13:18:43.832446+0000","last_fullsized":"2024-03-23T13:18:43.832446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:06:19.529125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670887+0000","last_change":"2024-03-23T13:17:41.247685+0000","last_active":"2024-03-23T13:18:43.670887+0000","last_peered":"2024-03-23T13:18:43.670887+0000","last_clean":"2024-03-23T13:18:43.670887+0000","last_became_active":"2024-03-23T13:17:41.247389+0000","last_became_peered":"2024-03-23T13:17:41.247389+0000","last_unstale":"2024-03-23T13:18:43.670887+0000","last_undegraded":"2024-03-23T13:18:43.670887+0000","last_fullsized":"2024-03-23T13:18:43.670887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:53.258831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.679428+0000","last_change":"2024-03-23T13:18:43.679428+0000","last_active":"2024-03-23T13:18:43.679428+0000","last_peered":"2024-03-23T13:18:43.679428+0000","last_clean":"2024-03-23T13:18:43.679428+0000","last_became_active":"2024-03-23T13:18:43.675198+0000","last_became_peered":"2024-03-23T13:18:43.675198+0000","last_unstale":"2024-03-23T13:18:43.679428+0000","last_undegraded":"2024-03-23T13:18:43.679428+0000","last_fullsized":"2024-03-23T13:18:43.679428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:42.292393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670433+0000","last_change":"2024-03-23T13:17:41.239461+0000","last_active":"2024-03-23T13:18:43.670433+0000","last_peered":"2024-03-23T13:18:43.670433+0000","last_clean":"2024-03-23T13:18:43.670433+0000","last_became_active":"2024-03-23T13:17:41.239137+0000","last_became_peered":"2024-03-23T13:17:41.239137+0000","last_unstale":"2024-03-23T13:18:43.670433+0000","last_undegraded":"2024-03-23T13:18:43.670433+0000","last_fullsized":"2024-03-23T13:18:43.670433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:06:43.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879026+0000","last_change":"2024-03-23T13:17:40.237796+0000","last_active":"2024-03-23T13:18:44.879026+0000","last_peered":"2024-03-23T13:18:44.879026+0000","last_clean":"2024-03-23T13:18:44.879026+0000","last_became_active":"2024-03-23T13:17:40.236596+0000","last_became_peered":"2024-03-23T13:17:40.236596+0000","last_unstale":"2024-03-23T13:18:44.879026+0000","last_undegraded":"2024-03-23T13:18:44.879026+0000","last_fullsized":"2024-03-23T13:18:44.879026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:22.767459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673294+0000","last_change":"2024-03-23T13:17:41.250379+0000","last_active":"2024-03-23T13:18:43.673294+0000","last_peered":"2024-03-23T13:18:43.673294+0000","last_clean":"2024-03-23T13:18:43.673294+0000","last_became_active":"2024-03-23T13:17:41.244850+0000","last_became_peered":"2024-03-23T13:17:41.244850+0000","last_unstale":"2024-03-23T13:18:43.673294+0000","last_undegraded":"2024-03-23T13:18:43.673294+0000","last_fullsized":"2024-03-23T13:18:43.673294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:34:42.741865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674909+0000","last_change":"2024-03-23T13:17:40.234334+0000","last_active":"2024-03-23T13:18:43.674909+0000","last_peered":"2024-03-23T13:18:43.674909+0000","last_clean":"2024-03-23T13:18:43.674909+0000","last_became_active":"2024-03-23T13:17:40.233953+0000","last_became_peered":"2024-03-23T13:17:40.233953+0000","last_unstale":"2024-03-23T13:18:43.674909+0000","last_undegraded":"2024-03-23T13:18:43.674909+0000","last_fullsized":"2024-03-23T13:18:43.674909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:53:48.238091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879680+0000","last_change":"2024-03-23T13:17:41.247960+0000","last_active":"2024-03-23T13:18:44.879680+0000","last_peered":"2024-03-23T13:18:44.879680+0000","last_clean":"2024-03-23T13:18:44.879680+0000","last_became_active":"2024-03-23T13:17:41.247459+0000","last_became_peered":"2024-03-23T13:17:41.247459+0000","last_unstale":"2024-03-23T13:18:44.879680+0000","last_undegraded":"2024-03-23T13:18:44.879680+0000","last_fullsized":"2024-03-23T13:18:44.879680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:01.345486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670623+0000","last_change":"2024-03-23T13:17:40.237228+0000","last_active":"2024-03-23T13:18:43.670623+0000","last_peered":"2024-03-23T13:18:43.670623+0000","last_clean":"2024-03-23T13:18:43.670623+0000","last_became_active":"2024-03-23T13:17:40.236843+0000","last_became_peered":"2024-03-23T13:17:40.236843+0000","last_unstale":"2024-03-23T13:18:43.670623+0000","last_undegraded":"2024-03-23T13:18:43.670623+0000","last_fullsized":"2024-03-23T13:18:43.670623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:31:53.315272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671330+0000","last_change":"2024-03-23T13:17:41.256459+0000","last_active":"2024-03-23T13:18:43.671330+0000","last_peered":"2024-03-23T13:18:43.671330+0000","last_clean":"2024-03-23T13:18:43.671330+0000","last_became_active":"2024-03-23T13:17:41.256168+0000","last_became_peered":"2024-03-23T13:17:41.256168+0000","last_unstale":"2024-03-23T13:18:43.671330+0000","last_undegraded":"2024-03-23T13:18:43.671330+0000","last_fullsized":"2024-03-23T13:18:43.671330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:50.488371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:55.345221+0000","last_change":"2024-03-23T13:17:40.224068+0000","last_active":"2024-03-23T13:18:55.345221+0000","last_peered":"2024-03-23T13:18:55.345221+0000","last_clean":"2024-03-23T13:18:55.345221+0000","last_became_active":"2024-03-23T13:17:40.223744+0000","last_became_peered":"2024-03-23T13:17:40.223744+0000","last_unstale":"2024-03-23T13:18:55.345221+0000","last_undegraded":"2024-03-23T13:18:55.345221+0000","last_fullsized":"2024-03-23T13:18:55.345221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:08:56.187695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671445+0000","last_change":"2024-03-23T13:17:41.257665+0000","last_active":"2024-03-23T13:18:43.671445+0000","last_peered":"2024-03-23T13:18:43.671445+0000","last_clean":"2024-03-23T13:18:43.671445+0000","last_became_active":"2024-03-23T13:17:41.256943+0000","last_became_peered":"2024-03-23T13:17:41.256943+0000","last_unstale":"2024-03-23T13:18:43.671445+0000","last_undegraded":"2024-03-23T13:18:43.671445+0000","last_fullsized":"2024-03-23T13:18:43.671445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:12:21.896800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087411+0000","last_change":"2024-03-23T13:17:40.236231+0000","last_active":"2024-03-23T13:18:44.087411+0000","last_peered":"2024-03-23T13:18:44.087411+0000","last_clean":"2024-03-23T13:18:44.087411+0000","last_became_active":"2024-03-23T13:17:40.235880+0000","last_became_peered":"2024-03-23T13:17:40.235880+0000","last_unstale":"2024-03-23T13:18:44.087411+0000","last_undegraded":"2024-03-23T13:18:44.087411+0000","last_fullsized":"2024-03-23T13:18:44.087411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:38.400890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670510+0000","last_change":"2024-03-23T13:17:41.252326+0000","last_active":"2024-03-23T13:18:43.670510+0000","last_peered":"2024-03-23T13:18:43.670510+0000","last_clean":"2024-03-23T13:18:43.670510+0000","last_became_active":"2024-03-23T13:17:41.252023+0000","last_became_peered":"2024-03-23T13:17:41.252023+0000","last_unstale":"2024-03-23T13:18:43.670510+0000","last_undegraded":"2024-03-23T13:18:43.670510+0000","last_fullsized":"2024-03-23T13:18:43.670510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:18:51.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183612+0000","last_change":"2024-03-23T13:17:40.239647+0000","last_active":"2024-03-23T13:18:44.183612+0000","last_peered":"2024-03-23T13:18:44.183612+0000","last_clean":"2024-03-23T13:18:44.183612+0000","last_became_active":"2024-03-23T13:17:40.239318+0000","last_became_peered":"2024-03-23T13:17:40.239318+0000","last_unstale":"2024-03-23T13:18:44.183612+0000","last_undegraded":"2024-03-23T13:18:44.183612+0000","last_fullsized":"2024-03-23T13:18:44.183612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:56:03.354003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670682+0000","last_change":"2024-03-23T13:17:41.255446+0000","last_active":"2024-03-23T13:18:43.670682+0000","last_peered":"2024-03-23T13:18:43.670682+0000","last_clean":"2024-03-23T13:18:43.670682+0000","last_became_active":"2024-03-23T13:17:41.255135+0000","last_became_peered":"2024-03-23T13:17:41.255135+0000","last_unstale":"2024-03-23T13:18:43.670682+0000","last_undegraded":"2024-03-23T13:18:43.670682+0000","last_fullsized":"2024-03-23T13:18:43.670682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:47.639958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673353+0000","last_change":"2024-03-23T13:17:41.245392+0000","last_active":"2024-03-23T13:18:43.673353+0000","last_peered":"2024-03-23T13:18:43.673353+0000","last_clean":"2024-03-23T13:18:43.673353+0000","last_became_active":"2024-03-23T13:17:41.244439+0000","last_became_peered":"2024-03-23T13:17:41.244439+0000","last_unstale":"2024-03-23T13:18:43.673353+0000","last_undegraded":"2024-03-23T13:18:43.673353+0000","last_fullsized":"2024-03-23T13:18:43.673353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:01:50.028867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674995+0000","last_change":"2024-03-23T13:17:41.253214+0000","last_active":"2024-03-23T13:18:43.674995+0000","last_peered":"2024-03-23T13:18:43.674995+0000","last_clean":"2024-03-23T13:18:43.674995+0000","last_became_active":"2024-03-23T13:17:41.252892+0000","last_became_peered":"2024-03-23T13:17:41.252892+0000","last_unstale":"2024-03-23T13:18:43.674995+0000","last_undegraded":"2024-03-23T13:18:43.674995+0000","last_fullsized":"2024-03-23T13:18:43.674995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:24:15.475541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674875+0000","last_change":"2024-03-23T13:17:41.247561+0000","last_active":"2024-03-23T13:18:43.674875+0000","last_peered":"2024-03-23T13:18:43.674875+0000","last_clean":"2024-03-23T13:18:43.674875+0000","last_became_active":"2024-03-23T13:17:41.247238+0000","last_became_peered":"2024-03-23T13:17:41.247238+0000","last_unstale":"2024-03-23T13:18:43.674875+0000","last_undegraded":"2024-03-23T13:18:43.674875+0000","last_fullsized":"2024-03-23T13:18:43.674875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:37:19.301021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670382+0000","last_change":"2024-03-23T13:17:41.241607+0000","last_active":"2024-03-23T13:18:43.670382+0000","last_peered":"2024-03-23T13:18:43.670382+0000","last_clean":"2024-03-23T13:18:43.670382+0000","last_became_active":"2024-03-23T13:17:41.241285+0000","last_became_peered":"2024-03-23T13:17:41.241285+0000","last_unstale":"2024-03-23T13:18:43.670382+0000","last_undegraded":"2024-03-23T13:18:43.670382+0000","last_fullsized":"2024-03-23T13:18:43.670382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:34:35.800087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183389+0000","last_change":"2024-03-23T13:17:41.245423+0000","last_active":"2024-03-23T13:18:44.183389+0000","last_peered":"2024-03-23T13:18:44.183389+0000","last_clean":"2024-03-23T13:18:44.183389+0000","last_became_active":"2024-03-23T13:17:41.245051+0000","last_became_peered":"2024-03-23T13:17:41.245051+0000","last_unstale":"2024-03-23T13:18:44.183389+0000","last_undegraded":"2024-03-23T13:18:44.183389+0000","last_fullsized":"2024-03-23T13:18:44.183389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:10:23.191491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.676996+0000","last_change":"2024-03-23T13:18:43.676996+0000","last_active":"2024-03-23T13:18:43.676996+0000","last_peered":"2024-03-23T13:18:43.676996+0000","last_clean":"2024-03-23T13:18:43.676996+0000","last_became_active":"2024-03-23T13:18:43.676550+0000","last_became_peered":"2024-03-23T13:18:43.676550+0000","last_unstale":"2024-03-23T13:18:43.676996+0000","last_undegraded":"2024-03-23T13:18:43.676996+0000","last_fullsized":"2024-03-23T13:18:43.676996+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:13:12.408152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.676203+0000","last_change":"2024-03-23T13:17:41.248578+0000","last_active":"2024-03-23T13:18:43.676203+0000","last_peered":"2024-03-23T13:18:43.676203+0000","last_clean":"2024-03-23T13:18:43.676203+0000","last_became_active":"2024-03-23T13:17:41.248264+0000","last_became_peered":"2024-03-23T13:17:41.248264+0000","last_unstale":"2024-03-23T13:18:43.676203+0000","last_undegraded":"2024-03-23T13:18:43.676203+0000","last_fullsized":"2024-03-23T13:18:43.676203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:42.937816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879896+0000","last_change":"2024-03-23T13:17:41.246138+0000","last_active":"2024-03-23T13:18:44.879896+0000","last_peered":"2024-03-23T13:18:44.879896+0000","last_clean":"2024-03-23T13:18:44.879896+0000","last_became_active":"2024-03-23T13:17:41.245596+0000","last_became_peered":"2024-03-23T13:17:41.245596+0000","last_unstale":"2024-03-23T13:18:44.879896+0000","last_undegraded":"2024-03-23T13:18:44.879896+0000","last_fullsized":"2024-03-23T13:18:44.879896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:51:43.524257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087209+0000","last_change":"2024-03-23T13:17:41.246819+0000","last_active":"2024-03-23T13:18:44.087209+0000","last_peered":"2024-03-23T13:18:44.087209+0000","last_clean":"2024-03-23T13:18:44.087209+0000","last_became_active":"2024-03-23T13:17:41.246389+0000","last_became_peered":"2024-03-23T13:17:41.246389+0000","last_unstale":"2024-03-23T13:18:44.087209+0000","last_undegraded":"2024-03-23T13:18:44.087209+0000","last_fullsized":"2024-03-23T13:18:44.087209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:40.759387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879429+0000","last_change":"2024-03-23T13:17:41.254185+0000","last_active":"2024-03-23T13:18:44.879429+0000","last_peered":"2024-03-23T13:18:44.879429+0000","last_clean":"2024-03-23T13:18:44.879429+0000","last_became_active":"2024-03-23T13:17:41.253828+0000","last_became_peered":"2024-03-23T13:17:41.253828+0000","last_unstale":"2024-03-23T13:18:44.879429+0000","last_undegraded":"2024-03-23T13:18:44.879429+0000","last_fullsized":"2024-03-23T13:18:44.879429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:27.826109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671887+0000","last_change":"2024-03-23T13:17:41.257669+0000","last_active":"2024-03-23T13:18:43.671887+0000","last_peered":"2024-03-23T13:18:43.671887+0000","last_clean":"2024-03-23T13:18:43.671887+0000","last_became_active":"2024-03-23T13:17:41.257341+0000","last_became_peered":"2024-03-23T13:17:41.257341+0000","last_unstale":"2024-03-23T13:18:43.671887+0000","last_undegraded":"2024-03-23T13:18:43.671887+0000","last_fullsized":"2024-03-23T13:18:43.671887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:58:08.777906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183413+0000","last_change":"2024-03-23T13:17:41.249078+0000","last_active":"2024-03-23T13:18:44.183413+0000","last_peered":"2024-03-23T13:18:44.183413+0000","last_clean":"2024-03-23T13:18:44.183413+0000","last_became_active":"2024-03-23T13:17:41.247804+0000","last_became_peered":"2024-03-23T13:17:41.247804+0000","last_unstale":"2024-03-23T13:18:44.183413+0000","last_undegraded":"2024-03-23T13:18:44.183413+0000","last_fullsized":"2024-03-23T13:18:44.183413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:20:19.514109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673342+0000","last_change":"2024-03-23T13:17:41.242707+0000","last_active":"2024-03-23T13:18:43.673342+0000","last_peered":"2024-03-23T13:18:43.673342+0000","last_clean":"2024-03-23T13:18:43.673342+0000","last_became_active":"2024-03-23T13:17:41.242367+0000","last_became_peered":"2024-03-23T13:17:41.242367+0000","last_unstale":"2024-03-23T13:18:43.673342+0000","last_undegraded":"2024-03-23T13:18:43.673342+0000","last_fullsized":"2024-03-23T13:18:43.673342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:14:30.695624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249559,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63765,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":726,"num_read_kb":436,"num_write":235,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":191295,"data_compressed":5604,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383707,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":1104410,"data_compressed":37987,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":456359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":1049646,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":456076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":513550,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":457415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":514161,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":455901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1048933,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34482,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34482,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":58313,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":58924,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":58688,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T13:18:59.828 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-23T13:19:00.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:59 smithi155 ceph-mon[36157]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T13:19:00.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:19:00.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:18:59 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:19:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:19:00.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:59 smithi130 ceph-mon[36225]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T13:19:00.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:19:00.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:18:59 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:19:01.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:00 smithi155 ceph-mon[36157]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:19:01.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:00 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:19:01.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:00 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:19:01.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:00 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:19:01.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:00 smithi007 ceph-mon[32184]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:19:01.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:00 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:19:01.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:00 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:19:01.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:00 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:19:01.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:00 smithi130 ceph-mon[36225]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:19:01.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:00 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:19:01.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:00 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:19:01.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:00 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.746+0000 7fd0cb851700 1 -- 172.21.15.7:0/2537430110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4102c10 msgr2=0x7fd0c410f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.746+0000 7fd0cb851700 1 --2- 172.21.15.7:0/2537430110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4102c10 0x7fd0c410f050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00b0d0 tx=0x7fd0bc00b3e0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.746+0000 7fd0cb851700 1 -- 172.21.15.7:0/2537430110 shutdown_connections 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.746+0000 7fd0cb851700 1 --2- 172.21.15.7:0/2537430110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4102c10 0x7fd0c410f050 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.746+0000 7fd0cb851700 1 --2- 172.21.15.7:0/2537430110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0c4108230 0x7fd0c4108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.746+0000 7fd0cb851700 1 --2- 172.21.15.7:0/2537430110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0c4102250 0x7fd0c41026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.746+0000 7fd0cb851700 1 -- 172.21.15.7:0/2537430110 >> 172.21.15.7:0/2537430110 conn(0x7fd0c4079330 msgr2=0x7fd0c4079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.746+0000 7fd0cb851700 1 -- 172.21.15.7:0/2537430110 shutdown_connections 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.746+0000 7fd0cb851700 1 -- 172.21.15.7:0/2537430110 wait complete. 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.746+0000 7fd0cb851700 1 Processor -- start 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0cb851700 1 -- start start 2024-03-23T13:19:01.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0cb851700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0c4102250 0x7fd0c410e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0cb851700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4102c10 0x7fd0c410e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0cb851700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0c4108230 0x7fd0c41a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0cb851700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41a3c50 con 0x7fd0c4102c10 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0cb851700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41a3d90 con 0x7fd0c4102250 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0cb851700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41a3f50 con 0x7fd0c4108230 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c8dec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4102c10 0x7fd0c410e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c8dec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4102c10 0x7fd0c410e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52504/0 (socket says 172.21.15.7:52504) 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c8dec700 1 -- 172.21.15.7:0/3322399094 learned_addr learned my addr 172.21.15.7:0/3322399094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c95ed700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0c4102250 0x7fd0c410e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c9dee700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0c4108230 0x7fd0c41a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c8dec700 1 -- 172.21.15.7:0/3322399094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0c4108230 msgr2=0x7fd0c41a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c8dec700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0c4108230 0x7fd0c41a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c8dec700 1 -- 172.21.15.7:0/3322399094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0c4102250 msgr2=0x7fd0c410e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:01.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c8dec700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0c4102250 0x7fd0c410e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c8dec700 1 -- 172.21.15.7:0/3322399094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc009d00 con 0x7fd0c4102c10 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c95ed700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0c4102250 0x7fd0c410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.747+0000 7fd0c8dec700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4102c10 0x7fd0c410e7b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400e9a0 tx=0x7fd0b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.748+0000 7fd0ba7fc700 1 -- 172.21.15.7:0/3322399094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4009790 con 0x7fd0c4102c10 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.748+0000 7fd0ba7fc700 1 -- 172.21.15.7:0/3322399094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b4010430 con 0x7fd0c4102c10 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.748+0000 7fd0cb851700 1 -- 172.21.15.7:0/3322399094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c41a4230 con 0x7fd0c4102c10 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.748+0000 7fd0ba7fc700 1 -- 172.21.15.7:0/3322399094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4018680 con 0x7fd0c4102c10 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.748+0000 7fd0cb851700 1 -- 172.21.15.7:0/3322399094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c41abac0 con 0x7fd0c4102c10 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.749+0000 7fd0cb851700 1 -- 172.21.15.7:0/3322399094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c4068ec0 con 0x7fd0c4102c10 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.750+0000 7fd0ba7fc700 1 -- 172.21.15.7:0/3322399094 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fd0b4010910 con 0x7fd0c4102c10 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.750+0000 7fd0ba7fc700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd0b00a21e0 0x7fd0b00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.750+0000 7fd0c95ed700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd0b00a21e0 0x7fd0b00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.750+0000 7fd0ba7fc700 1 -- 172.21.15.7:0/3322399094 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd0b4014070 con 0x7fd0c4102c10 2024-03-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.751+0000 7fd0c95ed700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd0b00a21e0 0x7fd0b00a46a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0011490 tx=0x7fd0c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.755+0000 7fd0ba7fc700 1 -- 172.21.15.7:0/3322399094 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd0b4053280 con 0x7fd0c4102c10 2024-03-23T13:19:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.954+0000 7fd0cb851700 1 -- 172.21.15.7:0/3322399094 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0c41122a0 con 0x7fd0b00a21e0 2024-03-23T13:19:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.961+0000 7fd0ba7fc700 1 -- 172.21.15.7:0/3322399094 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258970 (secure 0 0 0) 0x7fd0c41122a0 con 0x7fd0b00a21e0 2024-03-23T13:19:01.965 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 -- 172.21.15.7:0/3322399094 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd0b00a21e0 msgr2=0x7fd0b00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd0b00a21e0 0x7fd0b00a46a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0011490 tx=0x7fd0c0009260 comp rx=0 tx=0).stop 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 -- 172.21.15.7:0/3322399094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4102c10 msgr2=0x7fd0c410e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4102c10 0x7fd0c410e7b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400e9a0 tx=0x7fd0b400ecb0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 -- 172.21.15.7:0/3322399094 shutdown_connections 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd0b00a21e0 0x7fd0b00a46a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0c4108230 0x7fd0c41a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0c4102250 0x7fd0c410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 --2- 172.21.15.7:0/3322399094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4102c10 0x7fd0c410e7b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 -- 172.21.15.7:0/3322399094 >> 172.21.15.7:0/3322399094 conn(0x7fd0c4079330 msgr2=0x7fd0c4100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 -- 172.21.15.7:0/3322399094 shutdown_connections 2024-03-23T13:19:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:01.964+0000 7fd0cb851700 1 -- 172.21.15.7:0/3322399094 wait complete. 2024-03-23T13:19:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-23T13:19:02.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:01 smithi155 ceph-mon[36157]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-23T13:19:02.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:01 smithi007 ceph-mon[32184]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-23T13:19:02.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:01 smithi130 ceph-mon[36225]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-23T13:19:02.646 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-03-23T13:19:01.943411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654367,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":877,"num_read_kb":550,"num_write":418,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373392,"kb_used_data":11688,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568432,"statfs":{"total":1151940427776,"available":1151558074368,"internally_reserved":0,"allocated":11968512,"data_stored":7425945,"data_compressed":113961,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":54,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002908"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673757+0000","last_change":"2024-03-23T13:17:41.250141+0000","last_active":"2024-03-23T13:18:43.673757+0000","last_peered":"2024-03-23T13:18:43.673757+0000","last_clean":"2024-03-23T13:18:43.673757+0000","last_became_active":"2024-03-23T13:17:41.249578+0000","last_became_peered":"2024-03-23T13:17:41.249578+0000","last_unstale":"2024-03-23T13:18:43.673757+0000","last_undegraded":"2024-03-23T13:18:43.673757+0000","last_fullsized":"2024-03-23T13:18:43.673757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:45.858553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670910+0000","last_change":"2024-03-23T13:17:41.251945+0000","last_active":"2024-03-23T13:18:43.670910+0000","last_peered":"2024-03-23T13:18:43.670910+0000","last_clean":"2024-03-23T13:18:43.670910+0000","last_became_active":"2024-03-23T13:17:41.251603+0000","last_became_peered":"2024-03-23T13:17:41.251603+0000","last_unstale":"2024-03-23T13:18:43.670910+0000","last_undegraded":"2024-03-23T13:18:43.670910+0000","last_fullsized":"2024-03-23T13:18:43.670910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:08:55.160949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670770+0000","last_change":"2024-03-23T13:17:41.254293+0000","last_active":"2024-03-23T13:18:43.670770+0000","last_peered":"2024-03-23T13:18:43.670770+0000","last_clean":"2024-03-23T13:18:43.670770+0000","last_became_active":"2024-03-23T13:17:41.253981+0000","last_became_peered":"2024-03-23T13:17:41.253981+0000","last_unstale":"2024-03-23T13:18:43.670770+0000","last_undegraded":"2024-03-23T13:18:43.670770+0000","last_fullsized":"2024-03-23T13:18:43.670770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:23:17.445169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.832344+0000","last_change":"2024-03-23T13:17:41.252125+0000","last_active":"2024-03-23T13:18:43.832344+0000","last_peered":"2024-03-23T13:18:43.832344+0000","last_clean":"2024-03-23T13:18:43.832344+0000","last_became_active":"2024-03-23T13:17:41.251819+0000","last_became_peered":"2024-03-23T13:17:41.251819+0000","last_unstale":"2024-03-23T13:18:43.832344+0000","last_undegraded":"2024-03-23T13:18:43.832344+0000","last_fullsized":"2024-03-23T13:18:43.832344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:31.559237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670723+0000","last_change":"2024-03-23T13:17:41.244653+0000","last_active":"2024-03-23T13:18:43.670723+0000","last_peered":"2024-03-23T13:18:43.670723+0000","last_clean":"2024-03-23T13:18:43.670723+0000","last_became_active":"2024-03-23T13:17:41.244322+0000","last_became_peered":"2024-03-23T13:17:41.244322+0000","last_unstale":"2024-03-23T13:18:43.670723+0000","last_undegraded":"2024-03-23T13:18:43.670723+0000","last_fullsized":"2024-03-23T13:18:43.670723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:55:19.419296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.676631+0000","last_change":"2024-03-23T13:18:43.676631+0000","last_active":"2024-03-23T13:18:43.676631+0000","last_peered":"2024-03-23T13:18:43.676631+0000","last_clean":"2024-03-23T13:18:43.676631+0000","last_became_active":"2024-03-23T13:18:43.676219+0000","last_became_peered":"2024-03-23T13:18:43.676219+0000","last_unstale":"2024-03-23T13:18:43.676631+0000","last_undegraded":"2024-03-23T13:18:43.676631+0000","last_fullsized":"2024-03-23T13:18:43.676631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:42:20.673553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670785+0000","last_change":"2024-03-23T13:17:41.250721+0000","last_active":"2024-03-23T13:18:43.670785+0000","last_peered":"2024-03-23T13:18:43.670785+0000","last_clean":"2024-03-23T13:18:43.670785+0000","last_became_active":"2024-03-23T13:17:41.250403+0000","last_became_peered":"2024-03-23T13:17:41.250403+0000","last_unstale":"2024-03-23T13:18:43.670785+0000","last_undegraded":"2024-03-23T13:18:43.670785+0000","last_fullsized":"2024-03-23T13:18:43.670785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:51:54.657066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.088565+0000","last_change":"2024-03-23T13:18:44.088565+0000","last_active":"2024-03-23T13:18:44.088565+0000","last_peered":"2024-03-23T13:18:44.088565+0000","last_clean":"2024-03-23T13:18:44.088565+0000","last_became_active":"2024-03-23T13:18:44.088216+0000","last_became_peered":"2024-03-23T13:18:44.088216+0000","last_unstale":"2024-03-23T13:18:44.088565+0000","last_undegraded":"2024-03-23T13:18:44.088565+0000","last_fullsized":"2024-03-23T13:18:44.088565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:30:40.102611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.672771+0000","last_change":"2024-03-23T13:17:41.253829+0000","last_active":"2024-03-23T13:18:43.672771+0000","last_peered":"2024-03-23T13:18:43.672771+0000","last_clean":"2024-03-23T13:18:43.672771+0000","last_became_active":"2024-03-23T13:17:41.253520+0000","last_became_peered":"2024-03-23T13:17:41.253520+0000","last_unstale":"2024-03-23T13:18:43.672771+0000","last_undegraded":"2024-03-23T13:18:43.672771+0000","last_fullsized":"2024-03-23T13:18:43.672771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:27:40.159325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.679268+0000","last_change":"2024-03-23T13:18:43.679268+0000","last_active":"2024-03-23T13:18:43.679268+0000","last_peered":"2024-03-23T13:18:43.679268+0000","last_clean":"2024-03-23T13:18:43.679268+0000","last_became_active":"2024-03-23T13:18:43.678825+0000","last_became_peered":"2024-03-23T13:18:43.678825+0000","last_unstale":"2024-03-23T13:18:43.679268+0000","last_undegraded":"2024-03-23T13:18:43.679268+0000","last_fullsized":"2024-03-23T13:18:43.679268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:55:22.999752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671435+0000","last_change":"2024-03-23T13:17:41.257957+0000","last_active":"2024-03-23T13:18:43.671435+0000","last_peered":"2024-03-23T13:18:43.671435+0000","last_clean":"2024-03-23T13:18:43.671435+0000","last_became_active":"2024-03-23T13:17:41.257651+0000","last_became_peered":"2024-03-23T13:17:41.257651+0000","last_unstale":"2024-03-23T13:18:43.671435+0000","last_undegraded":"2024-03-23T13:18:43.671435+0000","last_fullsized":"2024-03-23T13:18:43.671435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:42:33.120763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.832332+0000","last_change":"2024-03-23T13:17:41.254935+0000","last_active":"2024-03-23T13:18:43.832332+0000","last_peered":"2024-03-23T13:18:43.832332+0000","last_clean":"2024-03-23T13:18:43.832332+0000","last_became_active":"2024-03-23T13:17:41.254633+0000","last_became_peered":"2024-03-23T13:17:41.254633+0000","last_unstale":"2024-03-23T13:18:43.832332+0000","last_undegraded":"2024-03-23T13:18:43.832332+0000","last_fullsized":"2024-03-23T13:18:43.832332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:23:18.345884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183385+0000","last_change":"2024-03-23T13:17:41.233447+0000","last_active":"2024-03-23T13:18:44.183385+0000","last_peered":"2024-03-23T13:18:44.183385+0000","last_clean":"2024-03-23T13:18:44.183385+0000","last_became_active":"2024-03-23T13:17:41.233143+0000","last_became_peered":"2024-03-23T13:17:41.233143+0000","last_unstale":"2024-03-23T13:18:44.183385+0000","last_undegraded":"2024-03-23T13:18:44.183385+0000","last_fullsized":"2024-03-23T13:18:44.183385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:16:53.677024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670101+0000","last_change":"2024-03-23T13:17:41.242668+0000","last_active":"2024-03-23T13:18:43.670101+0000","last_peered":"2024-03-23T13:18:43.670101+0000","last_clean":"2024-03-23T13:18:43.670101+0000","last_became_active":"2024-03-23T13:17:41.242321+0000","last_became_peered":"2024-03-23T13:17:41.242321+0000","last_unstale":"2024-03-23T13:18:43.670101+0000","last_undegraded":"2024-03-23T13:18:43.670101+0000","last_fullsized":"2024-03-23T13:18:43.670101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:17.783814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.184349+0000","last_change":"2024-03-23T13:18:44.184349+0000","last_active":"2024-03-23T13:18:44.184349+0000","last_peered":"2024-03-23T13:18:44.184349+0000","last_clean":"2024-03-23T13:18:44.184349+0000","last_became_active":"2024-03-23T13:18:44.183981+0000","last_became_peered":"2024-03-23T13:18:44.183981+0000","last_unstale":"2024-03-23T13:18:44.184349+0000","last_undegraded":"2024-03-23T13:18:44.184349+0000","last_fullsized":"2024-03-23T13:18:44.184349+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:45:08.419580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671717+0000","last_change":"2024-03-23T13:17:41.253679+0000","last_active":"2024-03-23T13:18:43.671717+0000","last_peered":"2024-03-23T13:18:43.671717+0000","last_clean":"2024-03-23T13:18:43.671717+0000","last_became_active":"2024-03-23T13:17:41.253371+0000","last_became_peered":"2024-03-23T13:17:41.253371+0000","last_unstale":"2024-03-23T13:18:43.671717+0000","last_undegraded":"2024-03-23T13:18:43.671717+0000","last_fullsized":"2024-03-23T13:18:43.671717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:16:08.397446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673853+0000","last_change":"2024-03-23T13:17:41.249856+0000","last_active":"2024-03-23T13:18:43.673853+0000","last_peered":"2024-03-23T13:18:43.673853+0000","last_clean":"2024-03-23T13:18:43.673853+0000","last_became_active":"2024-03-23T13:17:41.249163+0000","last_became_peered":"2024-03-23T13:17:41.249163+0000","last_unstale":"2024-03-23T13:18:43.673853+0000","last_undegraded":"2024-03-23T13:18:43.673853+0000","last_fullsized":"2024-03-23T13:18:43.673853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:14.184859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673543+0000","last_change":"2024-03-23T13:17:41.250048+0000","last_active":"2024-03-23T13:18:43.673543+0000","last_peered":"2024-03-23T13:18:43.673543+0000","last_clean":"2024-03-23T13:18:43.673543+0000","last_became_active":"2024-03-23T13:17:41.249491+0000","last_became_peered":"2024-03-23T13:17:41.249491+0000","last_unstale":"2024-03-23T13:18:43.673543+0000","last_undegraded":"2024-03-23T13:18:43.673543+0000","last_fullsized":"2024-03-23T13:18:43.673543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:55:45.253522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.675991+0000","last_change":"2024-03-23T13:17:41.256615+0000","last_active":"2024-03-23T13:18:43.675991+0000","last_peered":"2024-03-23T13:18:43.675991+0000","last_clean":"2024-03-23T13:18:43.675991+0000","last_became_active":"2024-03-23T13:17:41.256287+0000","last_became_peered":"2024-03-23T13:17:41.256287+0000","last_unstale":"2024-03-23T13:18:43.675991+0000","last_undegraded":"2024-03-23T13:18:43.675991+0000","last_fullsized":"2024-03-23T13:18:43.675991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:31.686699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:56.346078+0000","last_change":"2024-03-23T13:17:40.231464+0000","last_active":"2024-03-23T13:18:56.346078+0000","last_peered":"2024-03-23T13:18:56.346078+0000","last_clean":"2024-03-23T13:18:56.346078+0000","last_became_active":"2024-03-23T13:17:40.230711+0000","last_became_peered":"2024-03-23T13:17:40.230711+0000","last_unstale":"2024-03-23T13:18:56.346078+0000","last_undegraded":"2024-03-23T13:18:56.346078+0000","last_fullsized":"2024-03-23T13:18:56.346078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:17.926187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879850+0000","last_change":"2024-03-23T13:17:41.251449+0000","last_active":"2024-03-23T13:18:44.879850+0000","last_peered":"2024-03-23T13:18:44.879850+0000","last_clean":"2024-03-23T13:18:44.879850+0000","last_became_active":"2024-03-23T13:17:41.251077+0000","last_became_peered":"2024-03-23T13:17:41.251077+0000","last_unstale":"2024-03-23T13:18:44.879850+0000","last_undegraded":"2024-03-23T13:18:44.879850+0000","last_fullsized":"2024-03-23T13:18:44.879850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:09.939244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879189+0000","last_change":"2024-03-23T13:17:40.233263+0000","last_active":"2024-03-23T13:18:44.879189+0000","last_peered":"2024-03-23T13:18:44.879189+0000","last_clean":"2024-03-23T13:18:44.879189+0000","last_became_active":"2024-03-23T13:17:40.232686+0000","last_became_peered":"2024-03-23T13:17:40.232686+0000","last_unstale":"2024-03-23T13:18:44.879189+0000","last_undegraded":"2024-03-23T13:18:44.879189+0000","last_fullsized":"2024-03-23T13:18:44.879189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:26.671088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671009+0000","last_change":"2024-03-23T13:17:41.250532+0000","last_active":"2024-03-23T13:18:43.671009+0000","last_peered":"2024-03-23T13:18:43.671009+0000","last_clean":"2024-03-23T13:18:43.671009+0000","last_became_active":"2024-03-23T13:17:41.250230+0000","last_became_peered":"2024-03-23T13:17:41.250230+0000","last_unstale":"2024-03-23T13:18:43.671009+0000","last_undegraded":"2024-03-23T13:18:43.671009+0000","last_fullsized":"2024-03-23T13:18:43.671009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:13:27.406827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.843761+0000","last_change":"2024-03-23T13:18:43.843761+0000","last_active":"2024-03-23T13:18:43.843761+0000","last_peered":"2024-03-23T13:18:43.843761+0000","last_clean":"2024-03-23T13:18:43.843761+0000","last_became_active":"2024-03-23T13:18:43.833436+0000","last_became_peered":"2024-03-23T13:18:43.833436+0000","last_unstale":"2024-03-23T13:18:43.843761+0000","last_undegraded":"2024-03-23T13:18:43.843761+0000","last_fullsized":"2024-03-23T13:18:43.843761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:49:01.749814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.672057+0000","last_change":"2024-03-23T13:17:41.247529+0000","last_active":"2024-03-23T13:18:43.672057+0000","last_peered":"2024-03-23T13:18:43.672057+0000","last_clean":"2024-03-23T13:18:43.672057+0000","last_became_active":"2024-03-23T13:17:41.247008+0000","last_became_peered":"2024-03-23T13:17:41.247008+0000","last_unstale":"2024-03-23T13:18:43.672057+0000","last_undegraded":"2024-03-23T13:18:43.672057+0000","last_fullsized":"2024-03-23T13:18:43.672057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:29:22.015602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671864+0000","last_change":"2024-03-23T13:17:40.233544+0000","last_active":"2024-03-23T13:18:43.671864+0000","last_peered":"2024-03-23T13:18:43.671864+0000","last_clean":"2024-03-23T13:18:43.671864+0000","last_became_active":"2024-03-23T13:17:40.233222+0000","last_became_peered":"2024-03-23T13:17:40.233222+0000","last_unstale":"2024-03-23T13:18:43.671864+0000","last_undegraded":"2024-03-23T13:18:43.671864+0000","last_fullsized":"2024-03-23T13:18:43.671864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:15:32.147020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087543+0000","last_change":"2024-03-23T13:17:41.245643+0000","last_active":"2024-03-23T13:18:44.087543+0000","last_peered":"2024-03-23T13:18:44.087543+0000","last_clean":"2024-03-23T13:18:44.087543+0000","last_became_active":"2024-03-23T13:17:41.245295+0000","last_became_peered":"2024-03-23T13:17:41.245295+0000","last_unstale":"2024-03-23T13:18:44.087543+0000","last_undegraded":"2024-03-23T13:18:44.087543+0000","last_fullsized":"2024-03-23T13:18:44.087543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:27:26.480225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:59.342463+0000","last_change":"2024-03-23T13:17:40.236473+0000","last_active":"2024-03-23T13:18:59.342463+0000","last_peered":"2024-03-23T13:18:59.342463+0000","last_clean":"2024-03-23T13:18:59.342463+0000","last_became_active":"2024-03-23T13:17:40.236114+0000","last_became_peered":"2024-03-23T13:17:40.236114+0000","last_unstale":"2024-03-23T13:18:59.342463+0000","last_undegraded":"2024-03-23T13:18:59.342463+0000","last_fullsized":"2024-03-23T13:18:59.342463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:37:33.949014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670614+0000","last_change":"2024-03-23T13:17:41.241904+0000","last_active":"2024-03-23T13:18:43.670614+0000","last_peered":"2024-03-23T13:18:43.670614+0000","last_clean":"2024-03-23T13:18:43.670614+0000","last_became_active":"2024-03-23T13:17:41.241605+0000","last_became_peered":"2024-03-23T13:17:41.241605+0000","last_unstale":"2024-03-23T13:18:43.670614+0000","last_undegraded":"2024-03-23T13:18:43.670614+0000","last_fullsized":"2024-03-23T13:18:43.670614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:05:43.782714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674124+0000","last_change":"2024-03-23T13:17:40.232399+0000","last_active":"2024-03-23T13:18:43.674124+0000","last_peered":"2024-03-23T13:18:43.674124+0000","last_clean":"2024-03-23T13:18:43.674124+0000","last_became_active":"2024-03-23T13:17:40.232071+0000","last_became_peered":"2024-03-23T13:17:40.232071+0000","last_unstale":"2024-03-23T13:18:43.674124+0000","last_undegraded":"2024-03-23T13:18:43.674124+0000","last_fullsized":"2024-03-23T13:18:43.674124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:34:35.839756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087321+0000","last_change":"2024-03-23T13:17:41.246698+0000","last_active":"2024-03-23T13:18:44.087321+0000","last_peered":"2024-03-23T13:18:44.087321+0000","last_clean":"2024-03-23T13:18:44.087321+0000","last_became_active":"2024-03-23T13:17:41.246351+0000","last_became_peered":"2024-03-23T13:17:41.246351+0000","last_unstale":"2024-03-23T13:18:44.087321+0000","last_undegraded":"2024-03-23T13:18:44.087321+0000","last_fullsized":"2024-03-23T13:18:44.087321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:35:43.068833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183499+0000","last_change":"2024-03-23T13:17:40.230550+0000","last_active":"2024-03-23T13:18:44.183499+0000","last_peered":"2024-03-23T13:18:44.183499+0000","last_clean":"2024-03-23T13:18:44.183499+0000","last_became_active":"2024-03-23T13:17:40.230215+0000","last_became_peered":"2024-03-23T13:17:40.230215+0000","last_unstale":"2024-03-23T13:18:44.183499+0000","last_undegraded":"2024-03-23T13:18:44.183499+0000","last_fullsized":"2024-03-23T13:18:44.183499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:48:52.998612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879253+0000","last_change":"2024-03-23T13:17:41.246704+0000","last_active":"2024-03-23T13:18:44.879253+0000","last_peered":"2024-03-23T13:18:44.879253+0000","last_clean":"2024-03-23T13:18:44.879253+0000","last_became_active":"2024-03-23T13:17:41.246405+0000","last_became_peered":"2024-03-23T13:17:41.246405+0000","last_unstale":"2024-03-23T13:18:44.879253+0000","last_undegraded":"2024-03-23T13:18:44.879253+0000","last_fullsized":"2024-03-23T13:18:44.879253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:23:10.404830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087283+0000","last_change":"2024-03-23T13:17:40.232444+0000","last_active":"2024-03-23T13:18:44.087283+0000","last_peered":"2024-03-23T13:18:44.087283+0000","last_clean":"2024-03-23T13:18:44.087283+0000","last_became_active":"2024-03-23T13:17:40.232142+0000","last_became_peered":"2024-03-23T13:17:40.232142+0000","last_unstale":"2024-03-23T13:18:44.087283+0000","last_undegraded":"2024-03-23T13:18:44.087283+0000","last_fullsized":"2024-03-23T13:18:44.087283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:32:58.171626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673241+0000","last_change":"2024-03-23T13:17:41.246894+0000","last_active":"2024-03-23T13:18:43.673241+0000","last_peered":"2024-03-23T13:18:43.673241+0000","last_clean":"2024-03-23T13:18:43.673241+0000","last_became_active":"2024-03-23T13:17:41.246537+0000","last_became_peered":"2024-03-23T13:17:41.246537+0000","last_unstale":"2024-03-23T13:18:43.673241+0000","last_undegraded":"2024-03-23T13:18:43.673241+0000","last_fullsized":"2024-03-23T13:18:43.673241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:58.878823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":163,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:57.339930+0000","last_change":"2024-03-23T13:17:40.231214+0000","last_active":"2024-03-23T13:18:57.339930+0000","last_peered":"2024-03-23T13:18:57.339930+0000","last_clean":"2024-03-23T13:18:57.339930+0000","last_became_active":"2024-03-23T13:17:40.230890+0000","last_became_peered":"2024-03-23T13:17:40.230890+0000","last_unstale":"2024-03-23T13:18:57.339930+0000","last_undegraded":"2024-03-23T13:18:57.339930+0000","last_fullsized":"2024-03-23T13:18:57.339930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:00:57.300516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.672582+0000","last_change":"2024-03-23T13:17:41.245560+0000","last_active":"2024-03-23T13:18:43.672582+0000","last_peered":"2024-03-23T13:18:43.672582+0000","last_clean":"2024-03-23T13:18:43.672582+0000","last_became_active":"2024-03-23T13:17:41.245192+0000","last_became_peered":"2024-03-23T13:17:41.245192+0000","last_unstale":"2024-03-23T13:18:43.672582+0000","last_undegraded":"2024-03-23T13:18:43.672582+0000","last_fullsized":"2024-03-23T13:18:43.672582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:07:02.080036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670535+0000","last_change":"2024-03-23T13:17:40.232603+0000","last_active":"2024-03-23T13:18:43.670535+0000","last_peered":"2024-03-23T13:18:43.670535+0000","last_clean":"2024-03-23T13:18:43.670535+0000","last_became_active":"2024-03-23T13:17:40.231970+0000","last_became_peered":"2024-03-23T13:17:40.231970+0000","last_unstale":"2024-03-23T13:18:43.670535+0000","last_undegraded":"2024-03-23T13:18:43.670535+0000","last_fullsized":"2024-03-23T13:18:43.670535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:14:22.383047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671001+0000","last_change":"2024-03-23T13:17:41.244255+0000","last_active":"2024-03-23T13:18:43.671001+0000","last_peered":"2024-03-23T13:18:43.671001+0000","last_clean":"2024-03-23T13:18:43.671001+0000","last_became_active":"2024-03-23T13:17:41.243925+0000","last_became_peered":"2024-03-23T13:17:41.243925+0000","last_unstale":"2024-03-23T13:18:43.671001+0000","last_undegraded":"2024-03-23T13:18:43.671001+0000","last_fullsized":"2024-03-23T13:18:43.671001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:57:03.115703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087570+0000","last_change":"2024-03-23T13:17:40.229160+0000","last_active":"2024-03-23T13:18:44.087570+0000","last_peered":"2024-03-23T13:18:44.087570+0000","last_clean":"2024-03-23T13:18:44.087570+0000","last_became_active":"2024-03-23T13:17:40.228846+0000","last_became_peered":"2024-03-23T13:17:40.228846+0000","last_unstale":"2024-03-23T13:18:44.087570+0000","last_undegraded":"2024-03-23T13:18:44.087570+0000","last_fullsized":"2024-03-23T13:18:44.087570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:25.517124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.672225+0000","last_change":"2024-03-23T13:17:41.247364+0000","last_active":"2024-03-23T13:18:43.672225+0000","last_peered":"2024-03-23T13:18:43.672225+0000","last_clean":"2024-03-23T13:18:43.672225+0000","last_became_active":"2024-03-23T13:17:41.246671+0000","last_became_peered":"2024-03-23T13:17:41.246671+0000","last_unstale":"2024-03-23T13:18:43.672225+0000","last_undegraded":"2024-03-23T13:18:43.672225+0000","last_fullsized":"2024-03-23T13:18:43.672225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:50:15.654396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'28","reported_seq":194,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:57.339796+0000","last_change":"2024-03-23T13:17:40.229805+0000","last_active":"2024-03-23T13:18:57.339796+0000","last_peered":"2024-03-23T13:18:57.339796+0000","last_clean":"2024-03-23T13:18:57.339796+0000","last_became_active":"2024-03-23T13:17:40.229438+0000","last_became_peered":"2024-03-23T13:17:40.229438+0000","last_unstale":"2024-03-23T13:18:57.339796+0000","last_undegraded":"2024-03-23T13:18:57.339796+0000","last_fullsized":"2024-03-23T13:18:57.339796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:21:11.121141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56561,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":30,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670810+0000","last_change":"2024-03-23T13:17:41.250106+0000","last_active":"2024-03-23T13:18:43.670810+0000","last_peered":"2024-03-23T13:18:43.670810+0000","last_clean":"2024-03-23T13:18:43.670810+0000","last_became_active":"2024-03-23T13:17:41.249725+0000","last_became_peered":"2024-03-23T13:17:41.249725+0000","last_unstale":"2024-03-23T13:18:43.670810+0000","last_undegraded":"2024-03-23T13:18:43.670810+0000","last_fullsized":"2024-03-23T13:18:43.670810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:07:20.243310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.675154+0000","last_change":"2024-03-23T13:17:40.234631+0000","last_active":"2024-03-23T13:18:43.675154+0000","last_peered":"2024-03-23T13:18:43.675154+0000","last_clean":"2024-03-23T13:18:43.675154+0000","last_became_active":"2024-03-23T13:17:40.234294+0000","last_became_peered":"2024-03-23T13:17:40.234294+0000","last_unstale":"2024-03-23T13:18:43.675154+0000","last_undegraded":"2024-03-23T13:18:43.675154+0000","last_fullsized":"2024-03-23T13:18:43.675154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:57.058252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183269+0000","last_change":"2024-03-23T13:17:41.247425+0000","last_active":"2024-03-23T13:18:44.183269+0000","last_peered":"2024-03-23T13:18:44.183269+0000","last_clean":"2024-03-23T13:18:44.183269+0000","last_became_active":"2024-03-23T13:17:41.247125+0000","last_became_peered":"2024-03-23T13:17:41.247125+0000","last_unstale":"2024-03-23T13:18:44.183269+0000","last_undegraded":"2024-03-23T13:18:44.183269+0000","last_fullsized":"2024-03-23T13:18:44.183269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:50.118666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087147+0000","last_change":"2024-03-23T13:17:40.236179+0000","last_active":"2024-03-23T13:18:44.087147+0000","last_peered":"2024-03-23T13:18:44.087147+0000","last_clean":"2024-03-23T13:18:44.087147+0000","last_became_active":"2024-03-23T13:17:40.235810+0000","last_became_peered":"2024-03-23T13:17:40.235810+0000","last_unstale":"2024-03-23T13:18:44.087147+0000","last_undegraded":"2024-03-23T13:18:44.087147+0000","last_fullsized":"2024-03-23T13:18:44.087147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:27:28.193164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671081+0000","last_change":"2024-03-23T13:17:41.252261+0000","last_active":"2024-03-23T13:18:43.671081+0000","last_peered":"2024-03-23T13:18:43.671081+0000","last_clean":"2024-03-23T13:18:43.671081+0000","last_became_active":"2024-03-23T13:17:41.251949+0000","last_became_peered":"2024-03-23T13:17:41.251949+0000","last_unstale":"2024-03-23T13:18:43.671081+0000","last_undegraded":"2024-03-23T13:18:43.671081+0000","last_fullsized":"2024-03-23T13:18:43.671081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:02:39.576999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:59.349509+0000","last_change":"2024-03-23T13:17:40.228738+0000","last_active":"2024-03-23T13:18:59.349509+0000","last_peered":"2024-03-23T13:18:59.349509+0000","last_clean":"2024-03-23T13:18:59.349509+0000","last_became_active":"2024-03-23T13:17:40.228424+0000","last_became_peered":"2024-03-23T13:17:40.228424+0000","last_unstale":"2024-03-23T13:18:59.349509+0000","last_undegraded":"2024-03-23T13:18:59.349509+0000","last_fullsized":"2024-03-23T13:18:59.349509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:51.506223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670265+0000","last_change":"2024-03-23T13:17:41.247155+0000","last_active":"2024-03-23T13:18:43.670265+0000","last_peered":"2024-03-23T13:18:43.670265+0000","last_clean":"2024-03-23T13:18:43.670265+0000","last_became_active":"2024-03-23T13:17:41.246849+0000","last_became_peered":"2024-03-23T13:17:41.246849+0000","last_unstale":"2024-03-23T13:18:43.670265+0000","last_undegraded":"2024-03-23T13:18:43.670265+0000","last_fullsized":"2024-03-23T13:18:43.670265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:23.311456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670433+0000","last_change":"2024-03-23T13:17:40.232420+0000","last_active":"2024-03-23T13:18:43.670433+0000","last_peered":"2024-03-23T13:18:43.670433+0000","last_clean":"2024-03-23T13:18:43.670433+0000","last_became_active":"2024-03-23T13:17:40.231940+0000","last_became_peered":"2024-03-23T13:17:40.231940+0000","last_unstale":"2024-03-23T13:18:43.670433+0000","last_undegraded":"2024-03-23T13:18:43.670433+0000","last_fullsized":"2024-03-23T13:18:43.670433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:46.300579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.086994+0000","last_change":"2024-03-23T13:17:41.249717+0000","last_active":"2024-03-23T13:18:44.086994+0000","last_peered":"2024-03-23T13:18:44.086994+0000","last_clean":"2024-03-23T13:18:44.086994+0000","last_became_active":"2024-03-23T13:17:41.249286+0000","last_became_peered":"2024-03-23T13:17:41.249286+0000","last_unstale":"2024-03-23T13:18:44.086994+0000","last_undegraded":"2024-03-23T13:18:44.086994+0000","last_fullsized":"2024-03-23T13:18:44.086994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:47:19.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670921+0000","last_change":"2024-03-23T13:17:40.229172+0000","last_active":"2024-03-23T13:18:43.670921+0000","last_peered":"2024-03-23T13:18:43.670921+0000","last_clean":"2024-03-23T13:18:43.670921+0000","last_became_active":"2024-03-23T13:17:40.228858+0000","last_became_peered":"2024-03-23T13:17:40.228858+0000","last_unstale":"2024-03-23T13:18:43.670921+0000","last_undegraded":"2024-03-23T13:18:43.670921+0000","last_fullsized":"2024-03-23T13:18:43.670921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:17.929936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673485+0000","last_change":"2024-03-23T13:17:41.253041+0000","last_active":"2024-03-23T13:18:43.673485+0000","last_peered":"2024-03-23T13:18:43.673485+0000","last_clean":"2024-03-23T13:18:43.673485+0000","last_became_active":"2024-03-23T13:17:41.252739+0000","last_became_peered":"2024-03-23T13:17:41.252739+0000","last_unstale":"2024-03-23T13:18:43.673485+0000","last_undegraded":"2024-03-23T13:18:43.673485+0000","last_fullsized":"2024-03-23T13:18:43.673485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:27:52.560688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673447+0000","last_change":"2024-03-23T13:17:40.228098+0000","last_active":"2024-03-23T13:18:43.673447+0000","last_peered":"2024-03-23T13:18:43.673447+0000","last_clean":"2024-03-23T13:18:43.673447+0000","last_became_active":"2024-03-23T13:17:40.227778+0000","last_became_peered":"2024-03-23T13:17:40.227778+0000","last_unstale":"2024-03-23T13:18:43.673447+0000","last_undegraded":"2024-03-23T13:18:43.673447+0000","last_fullsized":"2024-03-23T13:18:43.673447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:25:38.862329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674553+0000","last_change":"2024-03-23T13:17:41.242666+0000","last_active":"2024-03-23T13:18:43.674553+0000","last_peered":"2024-03-23T13:18:43.674553+0000","last_clean":"2024-03-23T13:18:43.674553+0000","last_became_active":"2024-03-23T13:17:41.242371+0000","last_became_peered":"2024-03-23T13:17:41.242371+0000","last_unstale":"2024-03-23T13:18:43.674553+0000","last_undegraded":"2024-03-23T13:18:43.674553+0000","last_fullsized":"2024-03-23T13:18:43.674553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:40.870090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:58.348441+0000","last_change":"2024-03-23T13:17:40.233558+0000","last_active":"2024-03-23T13:18:58.348441+0000","last_peered":"2024-03-23T13:18:58.348441+0000","last_clean":"2024-03-23T13:18:58.348441+0000","last_became_active":"2024-03-23T13:17:40.232112+0000","last_became_peered":"2024-03-23T13:17:40.232112+0000","last_unstale":"2024-03-23T13:18:58.348441+0000","last_undegraded":"2024-03-23T13:18:58.348441+0000","last_fullsized":"2024-03-23T13:18:58.348441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:37:43.971598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674339+0000","last_change":"2024-03-23T13:17:41.245804+0000","last_active":"2024-03-23T13:18:43.674339+0000","last_peered":"2024-03-23T13:18:43.674339+0000","last_clean":"2024-03-23T13:18:43.674339+0000","last_became_active":"2024-03-23T13:17:41.245509+0000","last_became_peered":"2024-03-23T13:17:41.245509+0000","last_unstale":"2024-03-23T13:18:43.674339+0000","last_undegraded":"2024-03-23T13:18:43.674339+0000","last_fullsized":"2024-03-23T13:18:43.674339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:39:55.735413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087382+0000","last_change":"2024-03-23T13:17:41.250553+0000","last_active":"2024-03-23T13:18:44.087382+0000","last_peered":"2024-03-23T13:18:44.087382+0000","last_clean":"2024-03-23T13:18:44.087382+0000","last_became_active":"2024-03-23T13:17:41.250140+0000","last_became_peered":"2024-03-23T13:17:41.250140+0000","last_unstale":"2024-03-23T13:18:44.087382+0000","last_undegraded":"2024-03-23T13:18:44.087382+0000","last_fullsized":"2024-03-23T13:18:44.087382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:28.088801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:59.349076+0000","last_change":"2024-03-23T13:17:40.239651+0000","last_active":"2024-03-23T13:18:59.349076+0000","last_peered":"2024-03-23T13:18:59.349076+0000","last_clean":"2024-03-23T13:18:59.349076+0000","last_became_active":"2024-03-23T13:17:40.239326+0000","last_became_peered":"2024-03-23T13:17:40.239326+0000","last_unstale":"2024-03-23T13:18:59.349076+0000","last_undegraded":"2024-03-23T13:18:59.349076+0000","last_fullsized":"2024-03-23T13:18:59.349076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:56:36.147882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879605+0000","last_change":"2024-03-23T13:17:41.253148+0000","last_active":"2024-03-23T13:18:44.879605+0000","last_peered":"2024-03-23T13:18:44.879605+0000","last_clean":"2024-03-23T13:18:44.879605+0000","last_became_active":"2024-03-23T13:17:41.252571+0000","last_became_peered":"2024-03-23T13:17:41.252571+0000","last_unstale":"2024-03-23T13:18:44.879605+0000","last_undegraded":"2024-03-23T13:18:44.879605+0000","last_fullsized":"2024-03-23T13:18:44.879605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:19:36.477844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.832199+0000","last_change":"2024-03-23T13:17:40.227457+0000","last_active":"2024-03-23T13:18:43.832199+0000","last_peered":"2024-03-23T13:18:43.832199+0000","last_clean":"2024-03-23T13:18:43.832199+0000","last_became_active":"2024-03-23T13:17:40.227149+0000","last_became_peered":"2024-03-23T13:17:40.227149+0000","last_unstale":"2024-03-23T13:18:43.832199+0000","last_undegraded":"2024-03-23T13:18:43.832199+0000","last_fullsized":"2024-03-23T13:18:43.832199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:33:20.606023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670329+0000","last_change":"2024-03-23T13:16:31.257961+0000","last_active":"2024-03-23T13:18:43.670329+0000","last_peered":"2024-03-23T13:18:43.670329+0000","last_clean":"2024-03-23T13:18:43.670329+0000","last_became_active":"2024-03-23T13:16:31.257444+0000","last_became_peered":"2024-03-23T13:16:31.257444+0000","last_unstale":"2024-03-23T13:18:43.670329+0000","last_undegraded":"2024-03-23T13:18:43.670329+0000","last_fullsized":"2024-03-23T13:18:43.670329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:13:08.847194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:13:08.847194+0000","last_clean_scrub_stamp":"2024-03-23T13:13:08.847194+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:35:07.006029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183325+0000","last_change":"2024-03-23T13:17:41.249443+0000","last_active":"2024-03-23T13:18:44.183325+0000","last_peered":"2024-03-23T13:18:44.183325+0000","last_clean":"2024-03-23T13:18:44.183325+0000","last_became_active":"2024-03-23T13:17:41.248960+0000","last_became_peered":"2024-03-23T13:17:41.248960+0000","last_unstale":"2024-03-23T13:18:44.183325+0000","last_undegraded":"2024-03-23T13:18:44.183325+0000","last_fullsized":"2024-03-23T13:18:44.183325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:35:36.819271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670430+0000","last_change":"2024-03-23T13:17:40.230268+0000","last_active":"2024-03-23T13:18:43.670430+0000","last_peered":"2024-03-23T13:18:43.670430+0000","last_clean":"2024-03-23T13:18:43.670430+0000","last_became_active":"2024-03-23T13:17:40.229955+0000","last_became_peered":"2024-03-23T13:17:40.229955+0000","last_unstale":"2024-03-23T13:18:43.670430+0000","last_undegraded":"2024-03-23T13:18:43.670430+0000","last_fullsized":"2024-03-23T13:18:43.670430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:38.977844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.685224+0000","last_change":"2024-03-23T13:18:43.684867+0000","last_active":"2024-03-23T13:18:43.685224+0000","last_peered":"2024-03-23T13:18:43.685224+0000","last_clean":"2024-03-23T13:18:43.685224+0000","last_became_active":"2024-03-23T13:18:43.676582+0000","last_became_peered":"2024-03-23T13:18:43.676582+0000","last_unstale":"2024-03-23T13:18:43.685224+0000","last_undegraded":"2024-03-23T13:18:43.685224+0000","last_fullsized":"2024-03-23T13:18:43.685224+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:31:12.274326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183400+0000","last_change":"2024-03-23T13:17:41.248807+0000","last_active":"2024-03-23T13:18:44.183400+0000","last_peered":"2024-03-23T13:18:44.183400+0000","last_clean":"2024-03-23T13:18:44.183400+0000","last_became_active":"2024-03-23T13:17:41.248401+0000","last_became_peered":"2024-03-23T13:17:41.248401+0000","last_unstale":"2024-03-23T13:18:44.183400+0000","last_undegraded":"2024-03-23T13:18:44.183400+0000","last_fullsized":"2024-03-23T13:18:44.183400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:13:30.533996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673590+0000","last_change":"2024-03-23T13:17:40.233323+0000","last_active":"2024-03-23T13:18:43.673590+0000","last_peered":"2024-03-23T13:18:43.673590+0000","last_clean":"2024-03-23T13:18:43.673590+0000","last_became_active":"2024-03-23T13:17:40.233031+0000","last_became_peered":"2024-03-23T13:17:40.233031+0000","last_unstale":"2024-03-23T13:18:43.673590+0000","last_undegraded":"2024-03-23T13:18:43.673590+0000","last_fullsized":"2024-03-23T13:18:43.673590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:55:35.870633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879908+0000","last_change":"2024-03-23T13:17:41.243662+0000","last_active":"2024-03-23T13:18:44.879908+0000","last_peered":"2024-03-23T13:18:44.879908+0000","last_clean":"2024-03-23T13:18:44.879908+0000","last_became_active":"2024-03-23T13:17:41.243310+0000","last_became_peered":"2024-03-23T13:17:41.243310+0000","last_unstale":"2024-03-23T13:18:44.879908+0000","last_undegraded":"2024-03-23T13:18:44.879908+0000","last_fullsized":"2024-03-23T13:18:44.879908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:25:06.277185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.832446+0000","last_change":"2024-03-23T13:17:40.226698+0000","last_active":"2024-03-23T13:18:43.832446+0000","last_peered":"2024-03-23T13:18:43.832446+0000","last_clean":"2024-03-23T13:18:43.832446+0000","last_became_active":"2024-03-23T13:17:40.226305+0000","last_became_peered":"2024-03-23T13:17:40.226305+0000","last_unstale":"2024-03-23T13:18:43.832446+0000","last_undegraded":"2024-03-23T13:18:43.832446+0000","last_fullsized":"2024-03-23T13:18:43.832446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:06:19.529125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670887+0000","last_change":"2024-03-23T13:17:41.247685+0000","last_active":"2024-03-23T13:18:43.670887+0000","last_peered":"2024-03-23T13:18:43.670887+0000","last_clean":"2024-03-23T13:18:43.670887+0000","last_became_active":"2024-03-23T13:17:41.247389+0000","last_became_peered":"2024-03-23T13:17:41.247389+0000","last_unstale":"2024-03-23T13:18:43.670887+0000","last_undegraded":"2024-03-23T13:18:43.670887+0000","last_fullsized":"2024-03-23T13:18:43.670887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:38:53.258831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.679428+0000","last_change":"2024-03-23T13:18:43.679428+0000","last_active":"2024-03-23T13:18:43.679428+0000","last_peered":"2024-03-23T13:18:43.679428+0000","last_clean":"2024-03-23T13:18:43.679428+0000","last_became_active":"2024-03-23T13:18:43.675198+0000","last_became_peered":"2024-03-23T13:18:43.675198+0000","last_unstale":"2024-03-23T13:18:43.679428+0000","last_undegraded":"2024-03-23T13:18:43.679428+0000","last_fullsized":"2024-03-23T13:18:43.679428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:42.292393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670433+0000","last_change":"2024-03-23T13:17:41.239461+0000","last_active":"2024-03-23T13:18:43.670433+0000","last_peered":"2024-03-23T13:18:43.670433+0000","last_clean":"2024-03-23T13:18:43.670433+0000","last_became_active":"2024-03-23T13:17:41.239137+0000","last_became_peered":"2024-03-23T13:17:41.239137+0000","last_unstale":"2024-03-23T13:18:43.670433+0000","last_undegraded":"2024-03-23T13:18:43.670433+0000","last_fullsized":"2024-03-23T13:18:43.670433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:06:43.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879026+0000","last_change":"2024-03-23T13:17:40.237796+0000","last_active":"2024-03-23T13:18:44.879026+0000","last_peered":"2024-03-23T13:18:44.879026+0000","last_clean":"2024-03-23T13:18:44.879026+0000","last_became_active":"2024-03-23T13:17:40.236596+0000","last_became_peered":"2024-03-23T13:17:40.236596+0000","last_unstale":"2024-03-23T13:18:44.879026+0000","last_undegraded":"2024-03-23T13:18:44.879026+0000","last_fullsized":"2024-03-23T13:18:44.879026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:22.767459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673294+0000","last_change":"2024-03-23T13:17:41.250379+0000","last_active":"2024-03-23T13:18:43.673294+0000","last_peered":"2024-03-23T13:18:43.673294+0000","last_clean":"2024-03-23T13:18:43.673294+0000","last_became_active":"2024-03-23T13:17:41.244850+0000","last_became_peered":"2024-03-23T13:17:41.244850+0000","last_unstale":"2024-03-23T13:18:43.673294+0000","last_undegraded":"2024-03-23T13:18:43.673294+0000","last_fullsized":"2024-03-23T13:18:43.673294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:34:42.741865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674909+0000","last_change":"2024-03-23T13:17:40.234334+0000","last_active":"2024-03-23T13:18:43.674909+0000","last_peered":"2024-03-23T13:18:43.674909+0000","last_clean":"2024-03-23T13:18:43.674909+0000","last_became_active":"2024-03-23T13:17:40.233953+0000","last_became_peered":"2024-03-23T13:17:40.233953+0000","last_unstale":"2024-03-23T13:18:43.674909+0000","last_undegraded":"2024-03-23T13:18:43.674909+0000","last_fullsized":"2024-03-23T13:18:43.674909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:53:48.238091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879680+0000","last_change":"2024-03-23T13:17:41.247960+0000","last_active":"2024-03-23T13:18:44.879680+0000","last_peered":"2024-03-23T13:18:44.879680+0000","last_clean":"2024-03-23T13:18:44.879680+0000","last_became_active":"2024-03-23T13:17:41.247459+0000","last_became_peered":"2024-03-23T13:17:41.247459+0000","last_unstale":"2024-03-23T13:18:44.879680+0000","last_undegraded":"2024-03-23T13:18:44.879680+0000","last_fullsized":"2024-03-23T13:18:44.879680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:32:01.345486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670623+0000","last_change":"2024-03-23T13:17:40.237228+0000","last_active":"2024-03-23T13:18:43.670623+0000","last_peered":"2024-03-23T13:18:43.670623+0000","last_clean":"2024-03-23T13:18:43.670623+0000","last_became_active":"2024-03-23T13:17:40.236843+0000","last_became_peered":"2024-03-23T13:17:40.236843+0000","last_unstale":"2024-03-23T13:18:43.670623+0000","last_undegraded":"2024-03-23T13:18:43.670623+0000","last_fullsized":"2024-03-23T13:18:43.670623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:31:53.315272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671330+0000","last_change":"2024-03-23T13:17:41.256459+0000","last_active":"2024-03-23T13:18:43.671330+0000","last_peered":"2024-03-23T13:18:43.671330+0000","last_clean":"2024-03-23T13:18:43.671330+0000","last_became_active":"2024-03-23T13:17:41.256168+0000","last_became_peered":"2024-03-23T13:17:41.256168+0000","last_unstale":"2024-03-23T13:18:43.671330+0000","last_undegraded":"2024-03-23T13:18:43.671330+0000","last_fullsized":"2024-03-23T13:18:43.671330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:50.488371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:19:00.350578+0000","last_change":"2024-03-23T13:17:40.224068+0000","last_active":"2024-03-23T13:19:00.350578+0000","last_peered":"2024-03-23T13:19:00.350578+0000","last_clean":"2024-03-23T13:19:00.350578+0000","last_became_active":"2024-03-23T13:17:40.223744+0000","last_became_peered":"2024-03-23T13:17:40.223744+0000","last_unstale":"2024-03-23T13:19:00.350578+0000","last_undegraded":"2024-03-23T13:19:00.350578+0000","last_fullsized":"2024-03-23T13:19:00.350578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:08:56.187695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671445+0000","last_change":"2024-03-23T13:17:41.257665+0000","last_active":"2024-03-23T13:18:43.671445+0000","last_peered":"2024-03-23T13:18:43.671445+0000","last_clean":"2024-03-23T13:18:43.671445+0000","last_became_active":"2024-03-23T13:17:41.256943+0000","last_became_peered":"2024-03-23T13:17:41.256943+0000","last_unstale":"2024-03-23T13:18:43.671445+0000","last_undegraded":"2024-03-23T13:18:43.671445+0000","last_fullsized":"2024-03-23T13:18:43.671445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:12:21.896800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087411+0000","last_change":"2024-03-23T13:17:40.236231+0000","last_active":"2024-03-23T13:18:44.087411+0000","last_peered":"2024-03-23T13:18:44.087411+0000","last_clean":"2024-03-23T13:18:44.087411+0000","last_became_active":"2024-03-23T13:17:40.235880+0000","last_became_peered":"2024-03-23T13:17:40.235880+0000","last_unstale":"2024-03-23T13:18:44.087411+0000","last_undegraded":"2024-03-23T13:18:44.087411+0000","last_fullsized":"2024-03-23T13:18:44.087411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:38.400890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670510+0000","last_change":"2024-03-23T13:17:41.252326+0000","last_active":"2024-03-23T13:18:43.670510+0000","last_peered":"2024-03-23T13:18:43.670510+0000","last_clean":"2024-03-23T13:18:43.670510+0000","last_became_active":"2024-03-23T13:17:41.252023+0000","last_became_peered":"2024-03-23T13:17:41.252023+0000","last_unstale":"2024-03-23T13:18:43.670510+0000","last_undegraded":"2024-03-23T13:18:43.670510+0000","last_fullsized":"2024-03-23T13:18:43.670510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:18:51.018290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183612+0000","last_change":"2024-03-23T13:17:40.239647+0000","last_active":"2024-03-23T13:18:44.183612+0000","last_peered":"2024-03-23T13:18:44.183612+0000","last_clean":"2024-03-23T13:18:44.183612+0000","last_became_active":"2024-03-23T13:17:40.239318+0000","last_became_peered":"2024-03-23T13:17:40.239318+0000","last_unstale":"2024-03-23T13:18:44.183612+0000","last_undegraded":"2024-03-23T13:18:44.183612+0000","last_fullsized":"2024-03-23T13:18:44.183612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:39.201971+0000","last_clean_scrub_stamp":"2024-03-23T13:17:39.201971+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:56:03.354003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670682+0000","last_change":"2024-03-23T13:17:41.255446+0000","last_active":"2024-03-23T13:18:43.670682+0000","last_peered":"2024-03-23T13:18:43.670682+0000","last_clean":"2024-03-23T13:18:43.670682+0000","last_became_active":"2024-03-23T13:17:41.255135+0000","last_became_peered":"2024-03-23T13:17:41.255135+0000","last_unstale":"2024-03-23T13:18:43.670682+0000","last_undegraded":"2024-03-23T13:18:43.670682+0000","last_fullsized":"2024-03-23T13:18:43.670682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:47.639958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673353+0000","last_change":"2024-03-23T13:17:41.245392+0000","last_active":"2024-03-23T13:18:43.673353+0000","last_peered":"2024-03-23T13:18:43.673353+0000","last_clean":"2024-03-23T13:18:43.673353+0000","last_became_active":"2024-03-23T13:17:41.244439+0000","last_became_peered":"2024-03-23T13:17:41.244439+0000","last_unstale":"2024-03-23T13:18:43.673353+0000","last_undegraded":"2024-03-23T13:18:43.673353+0000","last_fullsized":"2024-03-23T13:18:43.673353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:01:50.028867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674995+0000","last_change":"2024-03-23T13:17:41.253214+0000","last_active":"2024-03-23T13:18:43.674995+0000","last_peered":"2024-03-23T13:18:43.674995+0000","last_clean":"2024-03-23T13:18:43.674995+0000","last_became_active":"2024-03-23T13:17:41.252892+0000","last_became_peered":"2024-03-23T13:17:41.252892+0000","last_unstale":"2024-03-23T13:18:43.674995+0000","last_undegraded":"2024-03-23T13:18:43.674995+0000","last_fullsized":"2024-03-23T13:18:43.674995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:24:15.475541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.674875+0000","last_change":"2024-03-23T13:17:41.247561+0000","last_active":"2024-03-23T13:18:43.674875+0000","last_peered":"2024-03-23T13:18:43.674875+0000","last_clean":"2024-03-23T13:18:43.674875+0000","last_became_active":"2024-03-23T13:17:41.247238+0000","last_became_peered":"2024-03-23T13:17:41.247238+0000","last_unstale":"2024-03-23T13:18:43.674875+0000","last_undegraded":"2024-03-23T13:18:43.674875+0000","last_fullsized":"2024-03-23T13:18:43.674875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:37:19.301021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.670382+0000","last_change":"2024-03-23T13:17:41.241607+0000","last_active":"2024-03-23T13:18:43.670382+0000","last_peered":"2024-03-23T13:18:43.670382+0000","last_clean":"2024-03-23T13:18:43.670382+0000","last_became_active":"2024-03-23T13:17:41.241285+0000","last_became_peered":"2024-03-23T13:17:41.241285+0000","last_unstale":"2024-03-23T13:18:43.670382+0000","last_undegraded":"2024-03-23T13:18:43.670382+0000","last_fullsized":"2024-03-23T13:18:43.670382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:34:35.800087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183389+0000","last_change":"2024-03-23T13:17:41.245423+0000","last_active":"2024-03-23T13:18:44.183389+0000","last_peered":"2024-03-23T13:18:44.183389+0000","last_clean":"2024-03-23T13:18:44.183389+0000","last_became_active":"2024-03-23T13:17:41.245051+0000","last_became_peered":"2024-03-23T13:17:41.245051+0000","last_unstale":"2024-03-23T13:18:44.183389+0000","last_undegraded":"2024-03-23T13:18:44.183389+0000","last_fullsized":"2024-03-23T13:18:44.183389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:10:23.191491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.676996+0000","last_change":"2024-03-23T13:18:43.676996+0000","last_active":"2024-03-23T13:18:43.676996+0000","last_peered":"2024-03-23T13:18:43.676996+0000","last_clean":"2024-03-23T13:18:43.676996+0000","last_became_active":"2024-03-23T13:18:43.676550+0000","last_became_peered":"2024-03-23T13:18:43.676550+0000","last_unstale":"2024-03-23T13:18:43.676996+0000","last_undegraded":"2024-03-23T13:18:43.676996+0000","last_fullsized":"2024-03-23T13:18:43.676996+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:13:12.408152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.676203+0000","last_change":"2024-03-23T13:17:41.248578+0000","last_active":"2024-03-23T13:18:43.676203+0000","last_peered":"2024-03-23T13:18:43.676203+0000","last_clean":"2024-03-23T13:18:43.676203+0000","last_became_active":"2024-03-23T13:17:41.248264+0000","last_became_peered":"2024-03-23T13:17:41.248264+0000","last_unstale":"2024-03-23T13:18:43.676203+0000","last_undegraded":"2024-03-23T13:18:43.676203+0000","last_fullsized":"2024-03-23T13:18:43.676203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:42.937816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879896+0000","last_change":"2024-03-23T13:17:41.246138+0000","last_active":"2024-03-23T13:18:44.879896+0000","last_peered":"2024-03-23T13:18:44.879896+0000","last_clean":"2024-03-23T13:18:44.879896+0000","last_became_active":"2024-03-23T13:17:41.245596+0000","last_became_peered":"2024-03-23T13:17:41.245596+0000","last_unstale":"2024-03-23T13:18:44.879896+0000","last_undegraded":"2024-03-23T13:18:44.879896+0000","last_fullsized":"2024-03-23T13:18:44.879896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:51:43.524257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.087209+0000","last_change":"2024-03-23T13:17:41.246819+0000","last_active":"2024-03-23T13:18:44.087209+0000","last_peered":"2024-03-23T13:18:44.087209+0000","last_clean":"2024-03-23T13:18:44.087209+0000","last_became_active":"2024-03-23T13:17:41.246389+0000","last_became_peered":"2024-03-23T13:17:41.246389+0000","last_unstale":"2024-03-23T13:18:44.087209+0000","last_undegraded":"2024-03-23T13:18:44.087209+0000","last_fullsized":"2024-03-23T13:18:44.087209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:40.759387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.879429+0000","last_change":"2024-03-23T13:17:41.254185+0000","last_active":"2024-03-23T13:18:44.879429+0000","last_peered":"2024-03-23T13:18:44.879429+0000","last_clean":"2024-03-23T13:18:44.879429+0000","last_became_active":"2024-03-23T13:17:41.253828+0000","last_became_peered":"2024-03-23T13:17:41.253828+0000","last_unstale":"2024-03-23T13:18:44.879429+0000","last_undegraded":"2024-03-23T13:18:44.879429+0000","last_fullsized":"2024-03-23T13:18:44.879429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:27.826109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.671887+0000","last_change":"2024-03-23T13:17:41.257669+0000","last_active":"2024-03-23T13:18:43.671887+0000","last_peered":"2024-03-23T13:18:43.671887+0000","last_clean":"2024-03-23T13:18:43.671887+0000","last_became_active":"2024-03-23T13:17:41.257341+0000","last_became_peered":"2024-03-23T13:17:41.257341+0000","last_unstale":"2024-03-23T13:18:43.671887+0000","last_undegraded":"2024-03-23T13:18:43.671887+0000","last_fullsized":"2024-03-23T13:18:43.671887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:58:08.777906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:44.183413+0000","last_change":"2024-03-23T13:17:41.249078+0000","last_active":"2024-03-23T13:18:44.183413+0000","last_peered":"2024-03-23T13:18:44.183413+0000","last_clean":"2024-03-23T13:18:44.183413+0000","last_became_active":"2024-03-23T13:17:41.247804+0000","last_became_peered":"2024-03-23T13:17:41.247804+0000","last_unstale":"2024-03-23T13:18:44.183413+0000","last_undegraded":"2024-03-23T13:18:44.183413+0000","last_fullsized":"2024-03-23T13:18:44.183413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:20:19.514109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T13:18:43.673342+0000","last_change":"2024-03-23T13:17:41.242707+0000","last_active":"2024-03-23T13:18:43.673342+0000","last_peered":"2024-03-23T13:18:43.673342+0000","last_clean":"2024-03-23T13:18:43.673342+0000","last_became_active":"2024-03-23T13:17:41.242367+0000","last_became_peered":"2024-03-23T13:17:41.242367+0000","last_unstale":"2024-03-23T13:18:43.673342+0000","last_undegraded":"2024-03-23T13:18:43.673342+0000","last_fullsized":"2024-03-23T13:18:43.673342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T13:17:40.210871+0000","last_clean_scrub_stamp":"2024-03-23T13:17:40.210871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:14:30.695624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249559,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63765,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":751,"num_read_kb":441,"num_write":235,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":191295,"data_compressed":5604,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":1104410,"data_compressed":37987,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":456359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":1049646,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":456076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":513550,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":457415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":514161,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":455901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1048933,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34482,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34482,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36119,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":58313,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":58924,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":58688,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T13:19:02.650 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-23T13:19:02.650 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-23T13:19:02.650 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-23T13:19:02.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-23T13:19:02.900 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:19:04.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:03 smithi155 ceph-mon[36157]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-03-23T13:19:04.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:03 smithi155 ceph-mon[36157]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:19:04.055 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:03 smithi007 ceph-mon[32184]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-03-23T13:19:04.055 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:03 smithi007 ceph-mon[32184]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:19:04.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:03 smithi130 ceph-mon[36225]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-03-23T13:19:04.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:03 smithi130 ceph-mon[36225]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T13:19:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.589+0000 7fd6f32d5700 1 -- 172.21.15.7:0/1727445269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6ec102c10 msgr2=0x7fd6ec10f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.589+0000 7fd6f32d5700 1 --2- 172.21.15.7:0/1727445269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6ec102c10 0x7fd6ec10f050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40099c0 tx=0x7fd6e4009cd0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.589+0000 7fd6f32d5700 1 -- 172.21.15.7:0/1727445269 shutdown_connections 2024-03-23T13:19:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.589+0000 7fd6f32d5700 1 --2- 172.21.15.7:0/1727445269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6ec102c10 0x7fd6ec10f050 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.589+0000 7fd6f32d5700 1 --2- 172.21.15.7:0/1727445269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6ec108230 0x7fd6ec108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.589+0000 7fd6f32d5700 1 --2- 172.21.15.7:0/1727445269 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6ec102250 0x7fd6ec1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.589+0000 7fd6f32d5700 1 -- 172.21.15.7:0/1727445269 >> 172.21.15.7:0/1727445269 conn(0x7fd6ec079330 msgr2=0x7fd6ec079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.589+0000 7fd6f32d5700 1 -- 172.21.15.7:0/1727445269 shutdown_connections 2024-03-23T13:19:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.589+0000 7fd6f32d5700 1 -- 172.21.15.7:0/1727445269 wait complete. 2024-03-23T13:19:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f32d5700 1 Processor -- start 2024-03-23T13:19:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f32d5700 1 -- start start 2024-03-23T13:19:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f32d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6ec102250 0x7fd6ec1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f32d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6ec102c10 0x7fd6ec1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f32d5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6ec108230 0x7fd6ec19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f32d5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec19ac70 con 0x7fd6ec102250 2024-03-23T13:19:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f32d5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec19adb0 con 0x7fd6ec108230 2024-03-23T13:19:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f32d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec19af70 con 0x7fd6ec102c10 2024-03-23T13:19:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f1071700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6ec102250 0x7fd6ec1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f1071700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6ec102250 0x7fd6ec1a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33548/0 (socket says 172.21.15.7:33548) 2024-03-23T13:19:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f1071700 1 -- 172.21.15.7:0/3223324900 learned_addr learned my addr 172.21.15.7:0/3223324900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f1872700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6ec108230 0x7fd6ec19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f0870700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6ec102c10 0x7fd6ec1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f1071700 1 -- 172.21.15.7:0/3223324900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6ec102c10 msgr2=0x7fd6ec1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f1071700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6ec102c10 0x7fd6ec1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f1071700 1 -- 172.21.15.7:0/3223324900 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6ec108230 msgr2=0x7fd6ec19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f1071700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6ec108230 0x7fd6ec19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f1071700 1 -- 172.21.15.7:0/3223324900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e40095f0 con 0x7fd6ec102250 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.590+0000 7fd6f1071700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6ec102250 0x7fd6ec1a0450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00ef10 tx=0x7fd6dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.591+0000 7fd6ea7fc700 1 -- 172.21.15.7:0/3223324900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00c9c0 con 0x7fd6ec102250 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.591+0000 7fd6f32d5700 1 -- 172.21.15.7:0/3223324900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ec19b1f0 con 0x7fd6ec102250 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.591+0000 7fd6ea7fc700 1 -- 172.21.15.7:0/3223324900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6dc00f040 con 0x7fd6ec102250 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.591+0000 7fd6ea7fc700 1 -- 172.21.15.7:0/3223324900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc014480 con 0x7fd6ec102250 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.591+0000 7fd6f32d5700 1 -- 172.21.15.7:0/3223324900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ec1a74f0 con 0x7fd6ec102250 2024-03-23T13:19:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.592+0000 7fd6f32d5700 1 -- 172.21.15.7:0/3223324900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ec068ec0 con 0x7fd6ec102250 2024-03-23T13:19:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.593+0000 7fd6ea7fc700 1 -- 172.21.15.7:0/3223324900 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fd6dc007720 con 0x7fd6ec102250 2024-03-23T13:19:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.593+0000 7fd6ea7fc700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd6d40a21e0 0x7fd6d40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.593+0000 7fd6f0870700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd6d40a21e0 0x7fd6d40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.593+0000 7fd6ea7fc700 1 -- 172.21.15.7:0/3223324900 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd6dc0ce710 con 0x7fd6ec102250 2024-03-23T13:19:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.594+0000 7fd6f0870700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd6d40a21e0 0x7fd6d40a46a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec079bc0 tx=0x7fd6e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.597+0000 7fd6ea7fc700 1 -- 172.21.15.7:0/3223324900 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd6dc097590 con 0x7fd6ec102250 2024-03-23T13:19:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.848+0000 7fd6f32d5700 1 -- 172.21.15.7:0/3223324900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd6ec112230 con 0x7fd6ec102250 2024-03-23T13:19:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.848+0000 7fd6ea7fc700 1 -- 172.21.15.7:0/3223324900 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd6dc095e60 con 0x7fd6ec102250 2024-03-23T13:19:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T13:19:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.850+0000 7fd6f32d5700 1 -- 172.21.15.7:0/3223324900 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd6d40a21e0 msgr2=0x7fd6d40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.850+0000 7fd6f32d5700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd6d40a21e0 0x7fd6d40a46a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec079bc0 tx=0x7fd6e0009260 comp rx=0 tx=0).stop 2024-03-23T13:19:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.850+0000 7fd6f32d5700 1 -- 172.21.15.7:0/3223324900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6ec102250 msgr2=0x7fd6ec1a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.850+0000 7fd6f32d5700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6ec102250 0x7fd6ec1a0450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00ef10 tx=0x7fd6dc00c3a0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.850+0000 7fd6f32d5700 1 -- 172.21.15.7:0/3223324900 shutdown_connections 2024-03-23T13:19:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.850+0000 7fd6f32d5700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd6d40a21e0 0x7fd6d40a46a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.850+0000 7fd6f32d5700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6ec108230 0x7fd6ec19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.850+0000 7fd6f32d5700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6ec102250 0x7fd6ec1a0450 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.850+0000 7fd6f32d5700 1 --2- 172.21.15.7:0/3223324900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6ec102c10 0x7fd6ec1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.850+0000 7fd6f32d5700 1 -- 172.21.15.7:0/3223324900 >> 172.21.15.7:0/3223324900 conn(0x7fd6ec079330 msgr2=0x7fd6ec0746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.851+0000 7fd6f32d5700 1 -- 172.21.15.7:0/3223324900 shutdown_connections 2024-03-23T13:19:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:04.851+0000 7fd6f32d5700 1 -- 172.21.15.7:0/3223324900 wait complete. 2024-03-23T13:19:05.484 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-23T13:19:05.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.592+0000 7eff12119640 1 -- 172.21.15.7:0/567305779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0c1032e0 msgr2=0x7eff0c109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.592+0000 7eff12119640 1 --2- 172.21.15.7:0/567305779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0c1032e0 0x7eff0c109b70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7eff04009ca0 tx=0x7eff0402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.592+0000 7eff12119640 1 -- 172.21.15.7:0/567305779 shutdown_connections 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.592+0000 7eff12119640 1 --2- 172.21.15.7:0/567305779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0c1032e0 0x7eff0c109b70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.592+0000 7eff12119640 1 --2- 172.21.15.7:0/567305779 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff0c102920 0x7eff0c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.592+0000 7eff12119640 1 --2- 172.21.15.7:0/567305779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff0c101730 0x7eff0c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.592+0000 7eff12119640 1 -- 172.21.15.7:0/567305779 >> 172.21.15.7:0/567305779 conn(0x7eff0c0fccc0 msgr2=0x7eff0c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.592+0000 7eff12119640 1 -- 172.21.15.7:0/567305779 shutdown_connections 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.592+0000 7eff12119640 1 -- 172.21.15.7:0/567305779 wait complete. 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff12119640 1 Processor -- start 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff12119640 1 -- start start 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff12119640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0c101730 0x7eff0c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff12119640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff0c102920 0x7eff0c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff12119640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff0c1032e0 0x7eff0c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff12119640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c19f250 con 0x7eff0c101730 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff12119640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c19f3c0 con 0x7eff0c102920 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff12119640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c19f580 con 0x7eff0c1032e0 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0c101730 0x7eff0c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0c101730 0x7eff0c197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33568/0 (socket says 172.21.15.7:33568) 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff0b7fe640 1 -- 172.21.15.7:0/266183650 learned_addr learned my addr 172.21.15.7:0/266183650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff0affd640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff0c102920 0x7eff0c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff0bfff640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff0c1032e0 0x7eff0c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff0b7fe640 1 -- 172.21.15.7:0/266183650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff0c1032e0 msgr2=0x7eff0c19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff0b7fe640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff0c1032e0 0x7eff0c19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff0b7fe640 1 -- 172.21.15.7:0/266183650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff0c102920 msgr2=0x7eff0c197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff0b7fe640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff0c102920 0x7eff0c197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.593+0000 7eff0b7fe640 1 -- 172.21.15.7:0/266183650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff04009900 con 0x7eff0c101730 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.594+0000 7eff0b7fe640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0c101730 0x7eff0c197760 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efefc00e9c0 tx=0x7efefc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.594+0000 7eff08ff9640 1 -- 172.21.15.7:0/266183650 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc0096e0 con 0x7eff0c101730 2024-03-23T13:19:05.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.594+0000 7eff08ff9640 1 -- 172.21.15.7:0/266183650 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efefc00cd80 con 0x7eff0c101730 2024-03-23T13:19:05.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.594+0000 7eff12119640 1 -- 172.21.15.7:0/266183650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c19f860 con 0x7eff0c101730 2024-03-23T13:19:05.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.594+0000 7eff08ff9640 1 -- 172.21.15.7:0/266183650 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc010430 con 0x7eff0c101730 2024-03-23T13:19:05.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.594+0000 7eff12119640 1 -- 172.21.15.7:0/266183650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c19fd20 con 0x7eff0c101730 2024-03-23T13:19:05.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.595+0000 7eff12119640 1 -- 172.21.15.7:0/266183650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0c108a50 con 0x7eff0c101730 2024-03-23T13:19:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.596+0000 7eff08ff9640 1 -- 172.21.15.7:0/266183650 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7efefc003680 con 0x7eff0c101730 2024-03-23T13:19:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.597+0000 7eff08ff9640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7efef40a2150 0x7efef40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.597+0000 7eff0affd640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7efef40a2150 0x7efef40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.597+0000 7eff08ff9640 1 -- 172.21.15.7:0/266183650 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7efefc014070 con 0x7eff0c101730 2024-03-23T13:19:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.597+0000 7eff0affd640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7efef40a2150 0x7efef40a4610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7eff0c198c70 tx=0x7eff00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.601+0000 7eff08ff9640 1 -- 172.21.15.7:0/266183650 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7efefc097030 con 0x7eff0c101730 2024-03-23T13:19:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.785+0000 7eff12119640 1 -- 172.21.15.7:0/266183650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff0c10d130 con 0x7eff0c101730 2024-03-23T13:19:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.786+0000 7eff08ff9640 1 -- 172.21.15.7:0/266183650 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22558 (secure 0 0 0) 0x7efefc09a6f0 con 0x7eff0c101730 2024-03-23T13:19:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:05.787 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:18:43.667397+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 -- 172.21.15.7:0/266183650 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7efef40a2150 msgr2=0x7efef40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7efef40a2150 0x7efef40a4610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7eff0c198c70 tx=0x7eff00005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 -- 172.21.15.7:0/266183650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0c101730 msgr2=0x7eff0c197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0c101730 0x7eff0c197760 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efefc00e9c0 tx=0x7efefc00ee80 comp rx=0 tx=0).stop 2024-03-23T13:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 -- 172.21.15.7:0/266183650 shutdown_connections 2024-03-23T13:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7efef40a2150 0x7efef40a4610 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff0c1032e0 0x7eff0c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff0c102920 0x7eff0c197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 --2- 172.21.15.7:0/266183650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0c101730 0x7eff0c197760 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 -- 172.21.15.7:0/266183650 >> 172.21.15.7:0/266183650 conn(0x7eff0c0fccc0 msgr2=0x7eff0c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:05.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 -- 172.21.15.7:0/266183650 shutdown_connections 2024-03-23T13:19:05.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.788+0000 7eff12119640 1 -- 172.21.15.7:0/266183650 wait complete. 2024-03-23T13:19:05.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T13:13:08.186719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T13:17:38.843887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T13:17:39.788489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T13:19:05.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T13:19:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.905+0000 7ffbae593640 1 -- 172.21.15.7:0/3831562450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba8102990 msgr2=0x7ffba8102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.905+0000 7ffbae593640 1 --2- 172.21.15.7:0/3831562450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba8102990 0x7ffba8102e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009900 tx=0x7ffb9c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.905+0000 7ffbae593640 1 -- 172.21.15.7:0/3831562450 shutdown_connections 2024-03-23T13:19:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.905+0000 7ffbae593640 1 --2- 172.21.15.7:0/3831562450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba8103350 0x7ffba8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.905+0000 7ffbae593640 1 --2- 172.21.15.7:0/3831562450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba8102990 0x7ffba8102e10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.905+0000 7ffbae593640 1 --2- 172.21.15.7:0/3831562450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffba81017a0 0x7ffba8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.905+0000 7ffbae593640 1 -- 172.21.15.7:0/3831562450 >> 172.21.15.7:0/3831562450 conn(0x7ffba80fcd30 msgr2=0x7ffba80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.905+0000 7ffbae593640 1 -- 172.21.15.7:0/3831562450 shutdown_connections 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.905+0000 7ffbae593640 1 -- 172.21.15.7:0/3831562450 wait complete. 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffbae593640 1 Processor -- start 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffbae593640 1 -- start start 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffbae593640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba81017a0 0x7ffba80754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffbae593640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba8102990 0x7ffba8073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffbae593640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffba8103350 0x7ffba8074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffbae593640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffba80747a0 con 0x7ffba81017a0 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffbae593640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffba8074910 con 0x7ffba8103350 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffbae593640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffba81a1810 con 0x7ffba8102990 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffba7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba81017a0 0x7ffba80754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffba7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba81017a0 0x7ffba80754c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33590/0 (socket says 172.21.15.7:33590) 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffba7fff640 1 -- 172.21.15.7:0/1020864393 learned_addr learned my addr 172.21.15.7:0/1020864393 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.906+0000 7ffba77fe640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba8102990 0x7ffba8073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffbacb30640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffba8103350 0x7ffba8074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffba7fff640 1 -- 172.21.15.7:0/1020864393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba8102990 msgr2=0x7ffba8073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffba7fff640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba8102990 0x7ffba8073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffba7fff640 1 -- 172.21.15.7:0/1020864393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffba8103350 msgr2=0x7ffba8074040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffba7fff640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffba8103350 0x7ffba8074040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffba7fff640 1 -- 172.21.15.7:0/1020864393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c009560 con 0x7ffba81017a0 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffba7fff640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba81017a0 0x7ffba80754c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800e9f0 tx=0x7ffb9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffba57fa640 1 -- 172.21.15.7:0/1020864393 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98009700 con 0x7ffba81017a0 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffba57fa640 1 -- 172.21.15.7:0/1020864393 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9800cd80 con 0x7ffba81017a0 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffbae593640 1 -- 172.21.15.7:0/1020864393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba81a1ac0 con 0x7ffba81017a0 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffba57fa640 1 -- 172.21.15.7:0/1020864393 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98010690 con 0x7ffba81017a0 2024-03-23T13:19:05.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.907+0000 7ffbae593640 1 -- 172.21.15.7:0/1020864393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba81a1fa0 con 0x7ffba81017a0 2024-03-23T13:19:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.908+0000 7ffbae593640 1 -- 172.21.15.7:0/1020864393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba8068500 con 0x7ffba81017a0 2024-03-23T13:19:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.909+0000 7ffba57fa640 1 -- 172.21.15.7:0/1020864393 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7ffb98003710 con 0x7ffba81017a0 2024-03-23T13:19:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.910+0000 7ffba57fa640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ffb900a2150 0x7ffb900a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.910+0000 7ffba77fe640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ffb900a2150 0x7ffb900a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.910+0000 7ffba57fa640 1 -- 172.21.15.7:0/1020864393 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ffb98014070 con 0x7ffba81017a0 2024-03-23T13:19:05.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.910+0000 7ffba77fe640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ffb900a2150 0x7ffb900a4610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009230 tx=0x7ffb9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:05.914+0000 7ffba57fa640 1 -- 172.21.15.7:0/1020864393 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ffb98096fc0 con 0x7ffba81017a0 2024-03-23T13:19:06.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:05 smithi155 ceph-mon[36157]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T13:19:06.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:05 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3223324900' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T13:19:06.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:05 smithi007 ceph-mon[32184]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T13:19:06.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:05 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3223324900' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T13:19:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.097+0000 7ffbae593640 1 -- 172.21.15.7:0/1020864393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffba810d1c0 con 0x7ffba81017a0 2024-03-23T13:19:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.097+0000 7ffba57fa640 1 -- 172.21.15.7:0/1020864393 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7ffb9809a680 con 0x7ffba81017a0 2024-03-23T13:19:06.098 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-23T13:19:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.099+0000 7ffbae593640 1 -- 172.21.15.7:0/1020864393 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ffb900a2150 msgr2=0x7ffb900a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.099+0000 7ffbae593640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ffb900a2150 0x7ffb900a4610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009230 tx=0x7ffb9c005fd0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.099+0000 7ffbae593640 1 -- 172.21.15.7:0/1020864393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba81017a0 msgr2=0x7ffba80754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.099+0000 7ffbae593640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba81017a0 0x7ffba80754c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800e9f0 tx=0x7ffb9800eeb0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.100+0000 7ffbae593640 1 -- 172.21.15.7:0/1020864393 shutdown_connections 2024-03-23T13:19:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.100+0000 7ffbae593640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ffb900a2150 0x7ffb900a4610 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.100+0000 7ffbae593640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffba8103350 0x7ffba8074040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.100+0000 7ffbae593640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba8102990 0x7ffba8073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.100+0000 7ffbae593640 1 --2- 172.21.15.7:0/1020864393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba81017a0 0x7ffba80754c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.100+0000 7ffbae593640 1 -- 172.21.15.7:0/1020864393 >> 172.21.15.7:0/1020864393 conn(0x7ffba80fcd30 msgr2=0x7ffba8105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.100+0000 7ffbae593640 1 -- 172.21.15.7:0/1020864393 shutdown_connections 2024-03-23T13:19:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.100+0000 7ffbae593640 1 -- 172.21.15.7:0/1020864393 wait complete. 2024-03-23T13:19:06.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T13:19:06.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:05 smithi130 ceph-mon[36225]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T13:19:06.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:05 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3223324900' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T13:19:06.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.216+0000 7fc10c08e640 1 -- 172.21.15.7:0/4004314034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1041073f0 msgr2=0x7fc104109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.216+0000 7fc10c08e640 1 --2- 172.21.15.7:0/4004314034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1041073f0 0x7fc104109840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc100009be0 tx=0x7fc10002bef0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.216+0000 7fc10c08e640 1 -- 172.21.15.7:0/4004314034 shutdown_connections 2024-03-23T13:19:06.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.216+0000 7fc10c08e640 1 --2- 172.21.15.7:0/4004314034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1041073f0 0x7fc104109840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.216+0000 7fc10c08e640 1 --2- 172.21.15.7:0/4004314034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104104ac0 0x7fc104106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.216+0000 7fc10c08e640 1 --2- 172.21.15.7:0/4004314034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104102190 0x7fc104104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.216+0000 7fc10c08e640 1 -- 172.21.15.7:0/4004314034 >> 172.21.15.7:0/4004314034 conn(0x7fc1040fbba0 msgr2=0x7fc1040fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:06.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc10c08e640 1 -- 172.21.15.7:0/4004314034 shutdown_connections 2024-03-23T13:19:06.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc10c08e640 1 -- 172.21.15.7:0/4004314034 wait complete. 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc10c08e640 1 Processor -- start 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc10c08e640 1 -- start start 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc10c08e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104102190 0x7fc1041977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc10c08e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104104ac0 0x7fc104197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc10c08e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1041073f0 0x7fc10419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc10c08e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc10419f250 con 0x7fc104104ac0 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc10c08e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc10419f3c0 con 0x7fc104102190 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc10c08e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc10419f580 con 0x7fc1041073f0 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc109629640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104104ac0 0x7fc104197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc109629640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104104ac0 0x7fc104197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33608/0 (socket says 172.21.15.7:33608) 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.217+0000 7fc109629640 1 -- 172.21.15.7:0/1735686368 learned_addr learned my addr 172.21.15.7:0/1735686368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc109e2a640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104102190 0x7fc1041977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc10a62b640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1041073f0 0x7fc10419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc109629640 1 -- 172.21.15.7:0/1735686368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1041073f0 msgr2=0x7fc10419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc109629640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1041073f0 0x7fc10419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc109629640 1 -- 172.21.15.7:0/1735686368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104102190 msgr2=0x7fc1041977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc109629640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104102190 0x7fc1041977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc109629640 1 -- 172.21.15.7:0/1735686368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc100009840 con 0x7fc104104ac0 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc109e2a640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104102190 0x7fc1041977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc109629640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104104ac0 0x7fc104197cf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400e9c0 tx=0x7fc0f400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc0faffd640 1 -- 172.21.15.7:0/1735686368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4009700 con 0x7fc104104ac0 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc0faffd640 1 -- 172.21.15.7:0/1735686368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0f4005500 con 0x7fc104104ac0 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc10c08e640 1 -- 172.21.15.7:0/1735686368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10419f860 con 0x7fc104104ac0 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc0faffd640 1 -- 172.21.15.7:0/1735686368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4010430 con 0x7fc104104ac0 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.218+0000 7fc10c08e640 1 -- 172.21.15.7:0/1735686368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10419fd70 con 0x7fc104104ac0 2024-03-23T13:19:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.219+0000 7fc10c08e640 1 -- 172.21.15.7:0/1735686368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1040fd690 con 0x7fc104104ac0 2024-03-23T13:19:06.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.220+0000 7fc0faffd640 1 -- 172.21.15.7:0/1735686368 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fc0f400ccc0 con 0x7fc104104ac0 2024-03-23T13:19:06.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.221+0000 7fc0faffd640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0f00a2150 0x7fc0f00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.221+0000 7fc109e2a640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0f00a2150 0x7fc0f00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.221+0000 7fc0faffd640 1 -- 172.21.15.7:0/1735686368 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc0f4014070 con 0x7fc104104ac0 2024-03-23T13:19:06.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.221+0000 7fc109e2a640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0f00a2150 0x7fc0f00a4610 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc005fd0 tx=0x7fc0fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.224+0000 7fc0faffd640 1 -- 172.21.15.7:0/1735686368 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc0f4097250 con 0x7fc104104ac0 2024-03-23T13:19:06.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.408+0000 7fc10c08e640 1 -- 172.21.15.7:0/1735686368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc10410d290 con 0x7fc104104ac0 2024-03-23T13:19:06.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.409+0000 7fc0faffd640 1 -- 172.21.15.7:0/1735686368 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fc10410d290 con 0x7fc104104ac0 2024-03-23T13:19:06.409 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-23T13:19:06.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 -- 172.21.15.7:0/1735686368 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0f00a2150 msgr2=0x7fc0f00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0f00a2150 0x7fc0f00a4610 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc005fd0 tx=0x7fc0fc009260 comp rx=0 tx=0).stop 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 -- 172.21.15.7:0/1735686368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104104ac0 msgr2=0x7fc104197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104104ac0 0x7fc104197cf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400e9c0 tx=0x7fc0f400edc0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 -- 172.21.15.7:0/1735686368 shutdown_connections 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0f00a2150 0x7fc0f00a4610 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1041073f0 0x7fc10419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104104ac0 0x7fc104197cf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 --2- 172.21.15.7:0/1735686368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104102190 0x7fc1041977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 -- 172.21.15.7:0/1735686368 >> 172.21.15.7:0/1735686368 conn(0x7fc1040fbba0 msgr2=0x7fc104102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 -- 172.21.15.7:0/1735686368 shutdown_connections 2024-03-23T13:19:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.411+0000 7fc10c08e640 1 -- 172.21.15.7:0/1735686368 wait complete. 2024-03-23T13:19:06.423 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T13:19:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.527+0000 7fa0add93640 1 -- 172.21.15.7:0/841092870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0a81017a0 msgr2=0x7fa0a8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.527+0000 7fa0add93640 1 --2- 172.21.15.7:0/841092870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0a81017a0 0x7fa0a8101bc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa098009ca0 tx=0x7fa09802c0f0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.527+0000 7fa0add93640 1 -- 172.21.15.7:0/841092870 shutdown_connections 2024-03-23T13:19:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.527+0000 7fa0add93640 1 --2- 172.21.15.7:0/841092870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0a8103350 0x7fa0a8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.527+0000 7fa0add93640 1 --2- 172.21.15.7:0/841092870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8102990 0x7fa0a8102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.527+0000 7fa0add93640 1 --2- 172.21.15.7:0/841092870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0a81017a0 0x7fa0a8101bc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.527+0000 7fa0add93640 1 -- 172.21.15.7:0/841092870 >> 172.21.15.7:0/841092870 conn(0x7fa0a80fcd30 msgr2=0x7fa0a80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.527+0000 7fa0add93640 1 -- 172.21.15.7:0/841092870 shutdown_connections 2024-03-23T13:19:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.527+0000 7fa0add93640 1 -- 172.21.15.7:0/841092870 wait complete. 2024-03-23T13:19:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0add93640 1 Processor -- start 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0add93640 1 -- start start 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0add93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0a81017a0 0x7fa0a81977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0add93640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8102990 0x7fa0a8197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0add93640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0a8103350 0x7fa0a819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0add93640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a819f2b0 con 0x7fa0a81017a0 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0add93640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a819f420 con 0x7fa0a8103350 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0add93640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a819f5e0 con 0x7fa0a8102990 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0a77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0a81017a0 0x7fa0a81977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0a77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0a81017a0 0x7fa0a81977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33630/0 (socket says 172.21.15.7:33630) 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0a77fe640 1 -- 172.21.15.7:0/3350867683 learned_addr learned my addr 172.21.15.7:0/3350867683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0a7fff640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0a8103350 0x7fa0a819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.528+0000 7fa0a6ffd640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8102990 0x7fa0a8197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0a77fe640 1 -- 172.21.15.7:0/3350867683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8102990 msgr2=0x7fa0a8197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0a77fe640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8102990 0x7fa0a8197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0a77fe640 1 -- 172.21.15.7:0/3350867683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0a8103350 msgr2=0x7fa0a819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0a77fe640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0a8103350 0x7fa0a819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0a77fe640 1 -- 172.21.15.7:0/3350867683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa098009900 con 0x7fa0a81017a0 2024-03-23T13:19:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0a77fe640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0a81017a0 0x7fa0a81977c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa098000f80 tx=0x7fa098034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0a4ff9640 1 -- 172.21.15.7:0/3350867683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980329b0 con 0x7fa0a81017a0 2024-03-23T13:19:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0a4ff9640 1 -- 172.21.15.7:0/3350867683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa098003e50 con 0x7fa0a81017a0 2024-03-23T13:19:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0add93640 1 -- 172.21.15.7:0/3350867683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a819f860 con 0x7fa0a81017a0 2024-03-23T13:19:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0a4ff9640 1 -- 172.21.15.7:0/3350867683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098040450 con 0x7fa0a81017a0 2024-03-23T13:19:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.529+0000 7fa0add93640 1 -- 172.21.15.7:0/3350867683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a819fd30 con 0x7fa0a81017a0 2024-03-23T13:19:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.530+0000 7fa0add93640 1 -- 172.21.15.7:0/3350867683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a8108ac0 con 0x7fa0a81017a0 2024-03-23T13:19:06.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.531+0000 7fa0a4ff9640 1 -- 172.21.15.7:0/3350867683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fa09800a040 con 0x7fa0a81017a0 2024-03-23T13:19:06.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.532+0000 7fa0a4ff9640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa0900a21d0 0x7fa0900a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.532+0000 7fa0a6ffd640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa0900a21d0 0x7fa0900a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.532+0000 7fa0a4ff9640 1 -- 172.21.15.7:0/3350867683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa0980f0e40 con 0x7fa0a81017a0 2024-03-23T13:19:06.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.532+0000 7fa0a6ffd640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa0900a21d0 0x7fa0900a4690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8198cd0 tx=0x7fa09c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.536+0000 7fa0a4ff9640 1 -- 172.21.15.7:0/3350867683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa0980b9df0 con 0x7fa0a81017a0 2024-03-23T13:19:06.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.716+0000 7fa0add93640 1 -- 172.21.15.7:0/3350867683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa0a802d160 con 0x7fa0a81017a0 2024-03-23T13:19:06.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.716+0000 7fa0a4ff9640 1 -- 172.21.15.7:0/3350867683 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fa0980440a0 con 0x7fa0a81017a0 2024-03-23T13:19:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 -- 172.21.15.7:0/3350867683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa0900a21d0 msgr2=0x7fa0900a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa0900a21d0 0x7fa0900a4690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8198cd0 tx=0x7fa09c005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 -- 172.21.15.7:0/3350867683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0a81017a0 msgr2=0x7fa0a81977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0a81017a0 0x7fa0a81977c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa098000f80 tx=0x7fa098034a90 comp rx=0 tx=0).stop 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 -- 172.21.15.7:0/3350867683 shutdown_connections 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa0900a21d0 0x7fa0900a4690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0a8103350 0x7fa0a819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8102990 0x7fa0a8197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 --2- 172.21.15.7:0/3350867683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0a81017a0 0x7fa0a81977c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 -- 172.21.15.7:0/3350867683 >> 172.21.15.7:0/3350867683 conn(0x7fa0a80fcd30 msgr2=0x7fa0a8105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 -- 172.21.15.7:0/3350867683 shutdown_connections 2024-03-23T13:19:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.718+0000 7fa0add93640 1 -- 172.21.15.7:0/3350867683 wait complete. 2024-03-23T13:19:06.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:19:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.832+0000 7fc353e3b640 1 -- 172.21.15.7:0/2640001698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c1006b0 msgr2=0x7fc34c100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.832+0000 7fc353e3b640 1 --2- 172.21.15.7:0/2640001698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c1006b0 0x7fc34c100ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc348009900 tx=0x7fc34802bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.832+0000 7fc353e3b640 1 -- 172.21.15.7:0/2640001698 shutdown_connections 2024-03-23T13:19:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.832+0000 7fc353e3b640 1 --2- 172.21.15.7:0/2640001698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc34c102260 0x7fc34c108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.832+0000 7fc353e3b640 1 --2- 172.21.15.7:0/2640001698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c1018a0 0x7fc34c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.832+0000 7fc353e3b640 1 --2- 172.21.15.7:0/2640001698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c1006b0 0x7fc34c100ad0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.832+0000 7fc353e3b640 1 -- 172.21.15.7:0/2640001698 >> 172.21.15.7:0/2640001698 conn(0x7fc34c0fbc40 msgr2=0x7fc34c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc353e3b640 1 -- 172.21.15.7:0/2640001698 shutdown_connections 2024-03-23T13:19:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc353e3b640 1 -- 172.21.15.7:0/2640001698 wait complete. 2024-03-23T13:19:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc353e3b640 1 Processor -- start 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc353e3b640 1 -- start start 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc353e3b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc34c1006b0 0x7fc34c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc353e3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c1018a0 0x7fc34c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc353e3b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102260 0x7fc34c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc353e3b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c19f310 con 0x7fc34c1018a0 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc353e3b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c19f480 con 0x7fc34c1006b0 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc353e3b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c19f640 con 0x7fc34c102260 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc3513d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c1018a0 0x7fc34c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc3513d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c1018a0 0x7fc34c197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33646/0 (socket says 172.21.15.7:33646) 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc3513d6640 1 -- 172.21.15.7:0/3602457950 learned_addr learned my addr 172.21.15.7:0/3602457950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc351bd7640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc34c1006b0 0x7fc34c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.833+0000 7fc3523d8640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102260 0x7fc34c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc3513d6640 1 -- 172.21.15.7:0/3602457950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102260 msgr2=0x7fc34c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc3513d6640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102260 0x7fc34c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc3513d6640 1 -- 172.21.15.7:0/3602457950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc34c1006b0 msgr2=0x7fc34c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc3513d6640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc34c1006b0 0x7fc34c197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc3513d6640 1 -- 172.21.15.7:0/3602457950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc348009560 con 0x7fc34c1018a0 2024-03-23T13:19:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc3513d6640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c1018a0 0x7fc34c197d60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00e9f0 tx=0x7fc33c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc342ffd640 1 -- 172.21.15.7:0/3602457950 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c009720 con 0x7fc34c1018a0 2024-03-23T13:19:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc342ffd640 1 -- 172.21.15.7:0/3602457950 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc33c00cd80 con 0x7fc34c1018a0 2024-03-23T13:19:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc353e3b640 1 -- 172.21.15.7:0/3602457950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34c19f920 con 0x7fc34c1018a0 2024-03-23T13:19:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc342ffd640 1 -- 172.21.15.7:0/3602457950 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c010690 con 0x7fc34c1018a0 2024-03-23T13:19:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.834+0000 7fc353e3b640 1 -- 172.21.15.7:0/3602457950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34c19fde0 con 0x7fc34c1018a0 2024-03-23T13:19:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.835+0000 7fc353e3b640 1 -- 172.21.15.7:0/3602457950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34c10c050 con 0x7fc34c1018a0 2024-03-23T13:19:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.836+0000 7fc342ffd640 1 -- 172.21.15.7:0/3602457950 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fc33c0046c0 con 0x7fc34c1018a0 2024-03-23T13:19:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.837+0000 7fc342ffd640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc3380a2150 0x7fc3380a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.837+0000 7fc351bd7640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc3380a2150 0x7fc3380a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.837+0000 7fc342ffd640 1 -- 172.21.15.7:0/3602457950 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc33c014070 con 0x7fc34c1018a0 2024-03-23T13:19:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.837+0000 7fc351bd7640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc3380a2150 0x7fc3380a4610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc348005ab0 tx=0x7fc348005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:06.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:06.841+0000 7fc342ffd640 1 -- 172.21.15.7:0/3602457950 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc33c097010 con 0x7fc34c1018a0 2024-03-23T13:19:07.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:06 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/266183650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:07.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:06 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1020864393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:07.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:06 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1735686368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:07.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:06 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3350867683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.062+0000 7fc353e3b640 1 -- 172.21.15.7:0/3602457950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc34c1a0000 con 0x7fc34c1018a0 2024-03-23T13:19:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.063+0000 7fc342ffd640 1 -- 172.21.15.7:0/3602457950 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11917 (secure 0 0 0) 0x7fc33c09a6d0 con 0x7fc34c1018a0 2024-03-23T13:19:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:07.064 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T13:17:40.597309+0000","modified":"2024-03-23T13:17:48.259350+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24475,"mds_3":24470,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6841/3415570481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3415570481},{"type":"v1","addr":"172.21.15.7:6841","nonce":3415570481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/791500055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":791500055},{"type":"v1","addr":"172.21.15.130:6839","nonce":791500055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24470,24463,24475,24503,24496]},"id":1}]} 2024-03-23T13:19:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-23T13:19:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 -- 172.21.15.7:0/3602457950 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc3380a2150 msgr2=0x7fc3380a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc3380a2150 0x7fc3380a4610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc348005ab0 tx=0x7fc348005fd0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 -- 172.21.15.7:0/3602457950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c1018a0 msgr2=0x7fc34c197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c1018a0 0x7fc34c197d60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00e9f0 tx=0x7fc33c00edf0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 -- 172.21.15.7:0/3602457950 shutdown_connections 2024-03-23T13:19:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc3380a2150 0x7fc3380a4610 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc34c102260 0x7fc34c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c1018a0 0x7fc34c197d60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 --2- 172.21.15.7:0/3602457950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc34c1006b0 0x7fc34c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 -- 172.21.15.7:0/3602457950 >> 172.21.15.7:0/3602457950 conn(0x7fc34c0fbc40 msgr2=0x7fc34c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 -- 172.21.15.7:0/3602457950 shutdown_connections 2024-03-23T13:19:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.065+0000 7fc353e3b640 1 -- 172.21.15.7:0/3602457950 wait complete. 2024-03-23T13:19:07.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:06 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/266183650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:07.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:06 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1020864393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:07.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:06 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1735686368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:07.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:06 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3350867683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:07.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:19:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 -- 172.21.15.7:0/1479126199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4204100630 msgr2=0x7f4204100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 --2- 172.21.15.7:0/1479126199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4204100630 0x7f4204100a50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009900 tx=0x7f41f802bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 -- 172.21.15.7:0/1479126199 shutdown_connections 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 --2- 172.21.15.7:0/1479126199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42041021e0 0x7f4204108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 --2- 172.21.15.7:0/1479126199 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4204101820 0x7f4204101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 --2- 172.21.15.7:0/1479126199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4204100630 0x7f4204100a50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 -- 172.21.15.7:0/1479126199 >> 172.21.15.7:0/1479126199 conn(0x7f42040fbba0 msgr2=0x7f42040fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 -- 172.21.15.7:0/1479126199 shutdown_connections 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 -- 172.21.15.7:0/1479126199 wait complete. 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 Processor -- start 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 -- start start 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4204100630 0x7f4204197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4204101820 0x7f4204197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42041021e0 0x7f420419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f420419f320 con 0x7f42041021e0 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f420419f490 con 0x7f4204101820 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.179+0000 7f420aace640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f420419f650 con 0x7f4204100630 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420906b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42041021e0 0x7f420419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420906b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42041021e0 0x7f420419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33668/0 (socket says 172.21.15.7:33668) 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420886a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4204100630 0x7f4204197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420906b640 1 -- 172.21.15.7:0/1398654369 learned_addr learned my addr 172.21.15.7:0/1398654369 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f4203fff640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4204101820 0x7f4204197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420906b640 1 -- 172.21.15.7:0/1398654369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4204100630 msgr2=0x7f4204197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420906b640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4204100630 0x7f4204197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420906b640 1 -- 172.21.15.7:0/1398654369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4204101820 msgr2=0x7f4204197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420906b640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4204101820 0x7f4204197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420906b640 1 -- 172.21.15.7:0/1398654369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f8009560 con 0x7f42041021e0 2024-03-23T13:19:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420906b640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42041021e0 0x7f420419ede0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00eb10 tx=0x7f41fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f4201ffb640 1 -- 172.21.15.7:0/1398654369 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc004b20 con 0x7f42041021e0 2024-03-23T13:19:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420aace640 1 -- 172.21.15.7:0/1398654369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420419f930 con 0x7f42041021e0 2024-03-23T13:19:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f4201ffb640 1 -- 172.21.15.7:0/1398654369 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41fc004c80 con 0x7f42041021e0 2024-03-23T13:19:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f4201ffb640 1 -- 172.21.15.7:0/1398654369 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc010650 con 0x7f42041021e0 2024-03-23T13:19:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.180+0000 7f420aace640 1 -- 172.21.15.7:0/1398654369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420419fdf0 con 0x7f42041021e0 2024-03-23T13:19:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.181+0000 7f420aace640 1 -- 172.21.15.7:0/1398654369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420410c090 con 0x7f42041021e0 2024-03-23T13:19:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.182+0000 7f4201ffb640 1 -- 172.21.15.7:0/1398654369 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f41fc005020 con 0x7f42041021e0 2024-03-23T13:19:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.183+0000 7f4201ffb640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f41f40a2150 0x7f41f40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.183+0000 7f4201ffb640 1 -- 172.21.15.7:0/1398654369 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f41fc014070 con 0x7f42041021e0 2024-03-23T13:19:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.183+0000 7f420886a640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f41f40a2150 0x7f41f40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.183+0000 7f420886a640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f41f40a2150 0x7f41f40a4610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f41f802c240 tx=0x7f41f802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:07.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.187+0000 7f4201ffb640 1 -- 172.21.15.7:0/1398654369 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f41fc097260 con 0x7f42041021e0 2024-03-23T13:19:07.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:06 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/266183650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:07.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:06 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1020864393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:07.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:06 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1735686368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:07.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:06 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3350867683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.370+0000 7f420aace640 1 -- 172.21.15.7:0/1398654369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42041a02f0 con 0x7f42041021e0 2024-03-23T13:19:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.372+0000 7f4201ffb640 1 -- 172.21.15.7:0/1398654369 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22558 (secure 0 0 0) 0x7f41fc09a920 con 0x7f42041021e0 2024-03-23T13:19:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:07.372 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:18:43.667397+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:19:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 -- 172.21.15.7:0/1398654369 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f41f40a2150 msgr2=0x7f41f40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f41f40a2150 0x7f41f40a4610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f41f802c240 tx=0x7f41f802c7e0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 -- 172.21.15.7:0/1398654369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42041021e0 msgr2=0x7f420419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42041021e0 0x7f420419ede0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00eb10 tx=0x7f41fc00c2d0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 -- 172.21.15.7:0/1398654369 shutdown_connections 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f41f40a2150 0x7f41f40a4610 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42041021e0 0x7f420419ede0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4204101820 0x7f4204197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 --2- 172.21.15.7:0/1398654369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4204100630 0x7f4204197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 -- 172.21.15.7:0/1398654369 >> 172.21.15.7:0/1398654369 conn(0x7f42040fbba0 msgr2=0x7f42040fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 -- 172.21.15.7:0/1398654369 shutdown_connections 2024-03-23T13:19:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.374+0000 7f420aace640 1 -- 172.21.15.7:0/1398654369 wait complete. 2024-03-23T13:19:07.386 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T13:13:08.186719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T13:17:38.843887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T13:17:39.788489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T13:19:07.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T13:19:07.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.491+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/2712806663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb41021c0 msgr2=0x7f2fb4108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.491+0000 7f2fbb1c9640 1 --2- 172.21.15.7:0/2712806663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb41021c0 0x7f2fb4108a50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009900 tx=0x7f2f9c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:07.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.491+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/2712806663 shutdown_connections 2024-03-23T13:19:07.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.491+0000 7f2fbb1c9640 1 --2- 172.21.15.7:0/2712806663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb41021c0 0x7f2fb4108a50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.491+0000 7f2fbb1c9640 1 --2- 172.21.15.7:0/2712806663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4101800 0x7f2fb4101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.491+0000 7f2fbb1c9640 1 --2- 172.21.15.7:0/2712806663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fb4100610 0x7f2fb4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.491+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/2712806663 >> 172.21.15.7:0/2712806663 conn(0x7f2fb40fbba0 msgr2=0x7f2fb40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:07.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.491+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/2712806663 shutdown_connections 2024-03-23T13:19:07.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.491+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/2712806663 wait complete. 2024-03-23T13:19:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fbb1c9640 1 Processor -- start 2024-03-23T13:19:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fbb1c9640 1 -- start start 2024-03-23T13:19:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fbb1c9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4100610 0x7f2fb4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fbb1c9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fb4101800 0x7f2fb4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fbb1c9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb41021c0 0x7f2fb419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fbb1c9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb419f240 con 0x7f2fb41021c0 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fbb1c9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb419f3b0 con 0x7f2fb4101800 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fbb1c9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb419f570 con 0x7f2fb4100610 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fb9766640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb41021c0 0x7f2fb419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fb9766640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb41021c0 0x7f2fb419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33688/0 (socket says 172.21.15.7:33688) 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fb9766640 1 -- 172.21.15.7:0/1330142636 learned_addr learned my addr 172.21.15.7:0/1330142636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fb8f65640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4100610 0x7f2fb4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fb3fff640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fb4101800 0x7f2fb4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fb9766640 1 -- 172.21.15.7:0/1330142636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4100610 msgr2=0x7f2fb4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fb9766640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4100610 0x7f2fb4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fb9766640 1 -- 172.21.15.7:0/1330142636 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fb4101800 msgr2=0x7f2fb4197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fb9766640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fb4101800 0x7f2fb4197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.492+0000 7f2fb9766640 1 -- 172.21.15.7:0/1330142636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9c009560 con 0x7f2fb41021c0 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.493+0000 7f2fb9766640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb41021c0 0x7f2fb419ed00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009230 tx=0x7f2f9c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.493+0000 7f2fb1ffb640 1 -- 172.21.15.7:0/1330142636 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c004440 con 0x7f2fb41021c0 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.493+0000 7f2fb1ffb640 1 -- 172.21.15.7:0/1330142636 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f9c0052d0 con 0x7f2fb41021c0 2024-03-23T13:19:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.493+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/1330142636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb419f7f0 con 0x7f2fb41021c0 2024-03-23T13:19:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.493+0000 7f2fb1ffb640 1 -- 172.21.15.7:0/1330142636 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c003cc0 con 0x7f2fb41021c0 2024-03-23T13:19:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.493+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/1330142636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb419fd50 con 0x7f2fb41021c0 2024-03-23T13:19:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.494+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/1330142636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb410c020 con 0x7f2fb41021c0 2024-03-23T13:19:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.495+0000 7f2fb1ffb640 1 -- 172.21.15.7:0/1330142636 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f2f9c005440 con 0x7f2fb41021c0 2024-03-23T13:19:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.496+0000 7f2fb1ffb640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2fa00a21d0 0x7f2fa00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.496+0000 7f2fb8f65640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2fa00a21d0 0x7f2fa00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.496+0000 7f2fb1ffb640 1 -- 172.21.15.7:0/1330142636 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2f9c0eff60 con 0x7f2fb41021c0 2024-03-23T13:19:07.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.496+0000 7f2fb8f65640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2fa00a21d0 0x7f2fa00a4690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4101660 tx=0x7f2fac009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:07.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.500+0000 7f2fb1ffb640 1 -- 172.21.15.7:0/1330142636 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2f9c0b8e90 con 0x7f2fb41021c0 2024-03-23T13:19:07.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.683+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/1330142636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2fb41a01f0 con 0x7f2fb41021c0 2024-03-23T13:19:07.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.683+0000 7f2fb1ffb640 1 -- 172.21.15.7:0/1330142636 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f2f9c0bc550 con 0x7f2fb41021c0 2024-03-23T13:19:07.684 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/1330142636 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2fa00a21d0 msgr2=0x7f2fa00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2fa00a21d0 0x7f2fa00a4690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4101660 tx=0x7f2fac009500 comp rx=0 tx=0).stop 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/1330142636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb41021c0 msgr2=0x7f2fb419ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb41021c0 0x7f2fb419ed00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009230 tx=0x7f2f9c004040 comp rx=0 tx=0).stop 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/1330142636 shutdown_connections 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2fa00a21d0 0x7f2fa00a4690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb41021c0 0x7f2fb419ed00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fb4101800 0x7f2fb4197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 --2- 172.21.15.7:0/1330142636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb4100610 0x7f2fb4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/1330142636 >> 172.21.15.7:0/1330142636 conn(0x7f2fb40fbba0 msgr2=0x7f2fb4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/1330142636 shutdown_connections 2024-03-23T13:19:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.685+0000 7f2fbb1c9640 1 -- 172.21.15.7:0/1330142636 wait complete. 2024-03-23T13:19:07.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T13:19:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.803+0000 7fb57fdf4640 1 -- 172.21.15.7:0/1882081385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5781021e0 msgr2=0x7fb578108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.803+0000 7fb57fdf4640 1 --2- 172.21.15.7:0/1882081385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5781021e0 0x7fb578108a70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb570009ca0 tx=0x7fb57002c0f0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.803+0000 7fb57fdf4640 1 -- 172.21.15.7:0/1882081385 shutdown_connections 2024-03-23T13:19:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.803+0000 7fb57fdf4640 1 --2- 172.21.15.7:0/1882081385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5781021e0 0x7fb578108a70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.803+0000 7fb57fdf4640 1 --2- 172.21.15.7:0/1882081385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb578101820 0x7fb578101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.803+0000 7fb57fdf4640 1 --2- 172.21.15.7:0/1882081385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb578100630 0x7fb578100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.803+0000 7fb57fdf4640 1 -- 172.21.15.7:0/1882081385 >> 172.21.15.7:0/1882081385 conn(0x7fb5780fbbc0 msgr2=0x7fb5780fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.803+0000 7fb57fdf4640 1 -- 172.21.15.7:0/1882081385 shutdown_connections 2024-03-23T13:19:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.803+0000 7fb57fdf4640 1 -- 172.21.15.7:0/1882081385 wait complete. 2024-03-23T13:19:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57fdf4640 1 Processor -- start 2024-03-23T13:19:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57fdf4640 1 -- start start 2024-03-23T13:19:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57fdf4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb578100630 0x7fb57806a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57fdf4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb578101820 0x7fb57819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57fdf4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5781021e0 0x7fb57819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57fdf4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb57819f230 con 0x7fb578100630 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57fdf4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb57819f3a0 con 0x7fb5781021e0 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57fdf4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb57819f560 con 0x7fb578101820 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57db90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb578100630 0x7fb57806a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57db90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb578100630 0x7fb57806a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33696/0 (socket says 172.21.15.7:33696) 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57d38f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb578101820 0x7fb57819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57db90640 1 -- 172.21.15.7:0/4291833758 learned_addr learned my addr 172.21.15.7:0/4291833758 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57e391640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5781021e0 0x7fb57819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57db90640 1 -- 172.21.15.7:0/4291833758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb578101820 msgr2=0x7fb57819c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57db90640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb578101820 0x7fb57819c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57db90640 1 -- 172.21.15.7:0/4291833758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5781021e0 msgr2=0x7fb57819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57db90640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5781021e0 0x7fb57819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.804+0000 7fb57db90640 1 -- 172.21.15.7:0/4291833758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb570009900 con 0x7fb578100630 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.805+0000 7fb57db90640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb578100630 0x7fb57806a660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb57400e9c0 tx=0x7fb57400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.805+0000 7fb56effd640 1 -- 172.21.15.7:0/4291833758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574009700 con 0x7fb578100630 2024-03-23T13:19:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.805+0000 7fb56effd640 1 -- 172.21.15.7:0/4291833758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57400cd80 con 0x7fb578100630 2024-03-23T13:19:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.805+0000 7fb57fdf4640 1 -- 172.21.15.7:0/4291833758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57819f840 con 0x7fb578100630 2024-03-23T13:19:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.805+0000 7fb56effd640 1 -- 172.21.15.7:0/4291833758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574010430 con 0x7fb578100630 2024-03-23T13:19:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.805+0000 7fb57fdf4640 1 -- 172.21.15.7:0/4291833758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57819fd50 con 0x7fb578100630 2024-03-23T13:19:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.806+0000 7fb57fdf4640 1 -- 172.21.15.7:0/4291833758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5781078f0 con 0x7fb578100630 2024-03-23T13:19:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.807+0000 7fb56effd640 1 -- 172.21.15.7:0/4291833758 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fb574003680 con 0x7fb578100630 2024-03-23T13:19:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.808+0000 7fb56effd640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb5640a2150 0x7fb5640a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.808+0000 7fb56effd640 1 -- 172.21.15.7:0/4291833758 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb574014070 con 0x7fb578100630 2024-03-23T13:19:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.808+0000 7fb57d38f640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb5640a2150 0x7fb5640a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.808+0000 7fb57d38f640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb5640a2150 0x7fb5640a4610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb57806a3d0 tx=0x7fb568005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:07.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.816+0000 7fb56effd640 1 -- 172.21.15.7:0/4291833758 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb574097050 con 0x7fb578100630 2024-03-23T13:19:08.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:07.999+0000 7fb57fdf4640 1 -- 172.21.15.7:0/4291833758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb57810c040 con 0x7fb578100630 2024-03-23T13:19:08.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.000+0000 7fb56effd640 1 -- 172.21.15.7:0/4291833758 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fb57409a710 con 0x7fb578100630 2024-03-23T13:19:08.000 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 -- 172.21.15.7:0/4291833758 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb5640a2150 msgr2=0x7fb5640a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb5640a2150 0x7fb5640a4610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb57806a3d0 tx=0x7fb568005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 -- 172.21.15.7:0/4291833758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb578100630 msgr2=0x7fb57806a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb578100630 0x7fb57806a660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb57400e9c0 tx=0x7fb57400ee80 comp rx=0 tx=0).stop 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 -- 172.21.15.7:0/4291833758 shutdown_connections 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb5640a2150 0x7fb5640a4610 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5781021e0 0x7fb57819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb578101820 0x7fb57819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 --2- 172.21.15.7:0/4291833758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb578100630 0x7fb57806a660 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 -- 172.21.15.7:0/4291833758 >> 172.21.15.7:0/4291833758 conn(0x7fb5780fbbc0 msgr2=0x7fb578104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 -- 172.21.15.7:0/4291833758 shutdown_connections 2024-03-23T13:19:08.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.002+0000 7fb57fdf4640 1 -- 172.21.15.7:0/4291833758 wait complete. 2024-03-23T13:19:08.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:07 smithi007 ceph-mon[32184]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-23T13:19:08.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:07 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3602457950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:08.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:07 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1398654369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:08.013 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:07 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1330142636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:08.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T13:19:08.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:07 smithi155 ceph-mon[36157]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-23T13:19:08.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:07 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3602457950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:08.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:07 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1398654369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:08.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:07 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1330142636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 -- 172.21.15.7:0/3885298607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414100670 msgr2=0x7f4414100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 --2- 172.21.15.7:0/3885298607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414100670 0x7f4414100a90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4408009900 tx=0x7f440802bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 -- 172.21.15.7:0/3885298607 shutdown_connections 2024-03-23T13:19:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 --2- 172.21.15.7:0/3885298607 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414102220 0x7f4414108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 --2- 172.21.15.7:0/3885298607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414101860 0x7f4414101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 --2- 172.21.15.7:0/3885298607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414100670 0x7f4414100a90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 -- 172.21.15.7:0/3885298607 >> 172.21.15.7:0/3885298607 conn(0x7f44140fbc00 msgr2=0x7f44140fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 -- 172.21.15.7:0/3885298607 shutdown_connections 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 -- 172.21.15.7:0/3885298607 wait complete. 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 Processor -- start 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 -- start start 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414100670 0x7f4414104600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414101860 0x7f4414102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102220 0x7f4414103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44141038e0 con 0x7f4414102220 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4414103a50 con 0x7f4414100670 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.119+0000 7f441b1e2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f44141a1840 con 0x7f4414101860 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f441977f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102220 0x7f4414103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f4418f7e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414100670 0x7f4414104600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f4418f7e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414100670 0x7f4414104600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:50876/0 (socket says 172.21.15.7:50876) 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f4418f7e640 1 -- 172.21.15.7:0/1548993275 learned_addr learned my addr 172.21.15.7:0/1548993275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f4413fff640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414101860 0x7f4414102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f441977f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102220 0x7f4414103180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33702/0 (socket says 172.21.15.7:33702) 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f441977f640 1 -- 172.21.15.7:0/1548993275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414101860 msgr2=0x7f4414102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f441977f640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414101860 0x7f4414102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f441977f640 1 -- 172.21.15.7:0/1548993275 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414100670 msgr2=0x7f4414104600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f441977f640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414100670 0x7f4414104600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f441977f640 1 -- 172.21.15.7:0/1548993275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4408009560 con 0x7f4414102220 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f4418f7e640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414100670 0x7f4414104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f441977f640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102220 0x7f4414103180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f440c00ea80 tx=0x7f440c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f4411ffb640 1 -- 172.21.15.7:0/1548993275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c00cea0 con 0x7f4414102220 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f4411ffb640 1 -- 172.21.15.7:0/1548993275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440c005020 con 0x7f4414102220 2024-03-23T13:19:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f441b1e2640 1 -- 172.21.15.7:0/1548993275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44141a1af0 con 0x7f4414102220 2024-03-23T13:19:08.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f4411ffb640 1 -- 172.21.15.7:0/1548993275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c010690 con 0x7f4414102220 2024-03-23T13:19:08.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.120+0000 7f441b1e2640 1 -- 172.21.15.7:0/1548993275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141a1fb0 con 0x7f4414102220 2024-03-23T13:19:08.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.122+0000 7f441b1e2640 1 -- 172.21.15.7:0/1548993275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441410c010 con 0x7f4414102220 2024-03-23T13:19:08.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.122+0000 7f4411ffb640 1 -- 172.21.15.7:0/1548993275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f440c005500 con 0x7f4414102220 2024-03-23T13:19:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.123+0000 7f4411ffb640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f44040a2150 0x7f44040a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.123+0000 7f4418f7e640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f44040a2150 0x7f44040a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.123+0000 7f4411ffb640 1 -- 172.21.15.7:0/1548993275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f440c014070 con 0x7f4414102220 2024-03-23T13:19:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.124+0000 7f4418f7e640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f44040a2150 0x7f44040a4610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f440802c840 tx=0x7f440802cd10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:08.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.127+0000 7f4411ffb640 1 -- 172.21.15.7:0/1548993275 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f440c097360 con 0x7f4414102220 2024-03-23T13:19:08.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:07 smithi130 ceph-mon[36225]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-23T13:19:08.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:07 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3602457950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:08.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:07 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1398654369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:08.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:07 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1330142636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.311+0000 7f441b1e2640 1 -- 172.21.15.7:0/1548993275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f44141a2410 con 0x7f4414102220 2024-03-23T13:19:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.311+0000 7f4411ffb640 1 -- 172.21.15.7:0/1548993275 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f440c005190 con 0x7f4414102220 2024-03-23T13:19:08.312 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-23T13:19:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.313+0000 7f441b1e2640 1 -- 172.21.15.7:0/1548993275 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f44040a2150 msgr2=0x7f44040a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.313+0000 7f441b1e2640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f44040a2150 0x7f44040a4610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f440802c840 tx=0x7f440802cd10 comp rx=0 tx=0).stop 2024-03-23T13:19:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.313+0000 7f441b1e2640 1 -- 172.21.15.7:0/1548993275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102220 msgr2=0x7f4414103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.313+0000 7f441b1e2640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102220 0x7f4414103180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f440c00ea80 tx=0x7f440c00ef40 comp rx=0 tx=0).stop 2024-03-23T13:19:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.313+0000 7f441b1e2640 1 -- 172.21.15.7:0/1548993275 shutdown_connections 2024-03-23T13:19:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.313+0000 7f441b1e2640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f44040a2150 0x7f44040a4610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.313+0000 7f441b1e2640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4414102220 0x7f4414103180 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.313+0000 7f441b1e2640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4414101860 0x7f4414102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.313+0000 7f441b1e2640 1 --2- 172.21.15.7:0/1548993275 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4414100670 0x7f4414104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.313+0000 7f441b1e2640 1 -- 172.21.15.7:0/1548993275 >> 172.21.15.7:0/1548993275 conn(0x7f44140fbc00 msgr2=0x7f44140fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.314+0000 7f441b1e2640 1 -- 172.21.15.7:0/1548993275 shutdown_connections 2024-03-23T13:19:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.314+0000 7f441b1e2640 1 -- 172.21.15.7:0/1548993275 wait complete. 2024-03-23T13:19:08.325 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:19:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.431+0000 7f768d250640 1 -- 172.21.15.7:0/3335680225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7688104ae0 msgr2=0x7f7688106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.431+0000 7f768d250640 1 --2- 172.21.15.7:0/3335680225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7688104ae0 0x7f7688106ed0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7670009900 tx=0x7f767002bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.431+0000 7f768d250640 1 -- 172.21.15.7:0/3335680225 shutdown_connections 2024-03-23T13:19:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.431+0000 7f768d250640 1 --2- 172.21.15.7:0/3335680225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7688107410 0x7f7688109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.431+0000 7f768d250640 1 --2- 172.21.15.7:0/3335680225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7688104ae0 0x7f7688106ed0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.431+0000 7f768d250640 1 --2- 172.21.15.7:0/3335680225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76881021b0 0x7f76881045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.431+0000 7f768d250640 1 -- 172.21.15.7:0/3335680225 >> 172.21.15.7:0/3335680225 conn(0x7f76880fbbc0 msgr2=0x7f76880fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.431+0000 7f768d250640 1 -- 172.21.15.7:0/3335680225 shutdown_connections 2024-03-23T13:19:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.431+0000 7f768d250640 1 -- 172.21.15.7:0/3335680225 wait complete. 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768d250640 1 Processor -- start 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768d250640 1 -- start start 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768d250640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76881021b0 0x7f76881977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768d250640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7688104ae0 0x7f7688197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768d250640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7688107410 0x7f768819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768d250640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f768819f2b0 con 0x7f7688107410 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768d250640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f768819f420 con 0x7f7688104ae0 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768d250640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f768819f5e0 con 0x7f76881021b0 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7688107410 0x7f768819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7688107410 0x7f768819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33714/0 (socket says 172.21.15.7:33714) 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f768759e640 1 -- 172.21.15.7:0/131084074 learned_addr learned my addr 172.21.15.7:0/131084074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.432+0000 7f7686d9d640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76881021b0 0x7f76881977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f768659c640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7688104ae0 0x7f7688197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f768759e640 1 -- 172.21.15.7:0/131084074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76881021b0 msgr2=0x7f76881977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f768759e640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76881021b0 0x7f76881977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f768759e640 1 -- 172.21.15.7:0/131084074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7688104ae0 msgr2=0x7f7688197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f768759e640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7688104ae0 0x7f7688197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f768759e640 1 -- 172.21.15.7:0/131084074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7670009560 con 0x7f7688107410 2024-03-23T13:19:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f768759e640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7688107410 0x7f768819ed70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f767800eaa0 tx=0x7f767800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f767ffff640 1 -- 172.21.15.7:0/131084074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767800cdb0 con 0x7f7688107410 2024-03-23T13:19:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f768d250640 1 -- 172.21.15.7:0/131084074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768819f8c0 con 0x7f7688107410 2024-03-23T13:19:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f767ffff640 1 -- 172.21.15.7:0/131084074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7678005020 con 0x7f7688107410 2024-03-23T13:19:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f767ffff640 1 -- 172.21.15.7:0/131084074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76780116b0 con 0x7f7688107410 2024-03-23T13:19:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.433+0000 7f768d250640 1 -- 172.21.15.7:0/131084074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768819fda0 con 0x7f7688107410 2024-03-23T13:19:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.434+0000 7f768d250640 1 -- 172.21.15.7:0/131084074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76880fd6b0 con 0x7f7688107410 2024-03-23T13:19:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.435+0000 7f767ffff640 1 -- 172.21.15.7:0/131084074 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f7678005520 con 0x7f7688107410 2024-03-23T13:19:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.436+0000 7f767ffff640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f76740a2150 0x7f76740a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.436+0000 7f7686d9d640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f76740a2150 0x7f76740a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.436+0000 7f767ffff640 1 -- 172.21.15.7:0/131084074 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f76780ce350 con 0x7f7688107410 2024-03-23T13:19:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.436+0000 7f7686d9d640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f76740a2150 0x7f76740a4610 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7680005d00 tx=0x7f768000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.439+0000 7f767ffff640 1 -- 172.21.15.7:0/131084074 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7678097280 con 0x7f7688107410 2024-03-23T13:19:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.661+0000 7f768d250640 1 -- 172.21.15.7:0/131084074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f768810d110 con 0x7f7688107410 2024-03-23T13:19:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.662+0000 7f767ffff640 1 -- 172.21.15.7:0/131084074 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11917 (secure 0 0 0) 0x7f767801f070 con 0x7f7688107410 2024-03-23T13:19:08.663 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:08.663 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T13:17:40.597309+0000","modified":"2024-03-23T13:17:48.259350+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24475,"mds_3":24470,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6841/3415570481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3415570481},{"type":"v1","addr":"172.21.15.7:6841","nonce":3415570481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/791500055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":791500055},{"type":"v1","addr":"172.21.15.130:6839","nonce":791500055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24470,24463,24475,24503,24496]},"id":1}]} 2024-03-23T13:19:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-23T13:19:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 -- 172.21.15.7:0/131084074 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f76740a2150 msgr2=0x7f76740a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f76740a2150 0x7f76740a4610 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7680005d00 tx=0x7f768000b410 comp rx=0 tx=0).stop 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 -- 172.21.15.7:0/131084074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7688107410 msgr2=0x7f768819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7688107410 0x7f768819ed70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f767800eaa0 tx=0x7f767800ef60 comp rx=0 tx=0).stop 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 -- 172.21.15.7:0/131084074 shutdown_connections 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f76740a2150 0x7f76740a4610 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7688107410 0x7f768819ed70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7688104ae0 0x7f7688197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 --2- 172.21.15.7:0/131084074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76881021b0 0x7f76881977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 -- 172.21.15.7:0/131084074 >> 172.21.15.7:0/131084074 conn(0x7f76880fbbc0 msgr2=0x7f7688102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 -- 172.21.15.7:0/131084074 shutdown_connections 2024-03-23T13:19:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.664+0000 7f768d250640 1 -- 172.21.15.7:0/131084074 wait complete. 2024-03-23T13:19:08.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:19:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 -- 172.21.15.7:0/4093433236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101750 msgr2=0x7f8414101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 --2- 172.21.15.7:0/4093433236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101750 0x7f8414101b70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8408009900 tx=0x7f840802bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 -- 172.21.15.7:0/4093433236 shutdown_connections 2024-03-23T13:19:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 --2- 172.21.15.7:0/4093433236 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8414103300 0x7f8414109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 --2- 172.21.15.7:0/4093433236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414102940 0x7f8414102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 --2- 172.21.15.7:0/4093433236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101750 0x7f8414101b70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 -- 172.21.15.7:0/4093433236 >> 172.21.15.7:0/4093433236 conn(0x7f84140fccc0 msgr2=0x7f84140ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 -- 172.21.15.7:0/4093433236 shutdown_connections 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 -- 172.21.15.7:0/4093433236 wait complete. 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 Processor -- start 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.780+0000 7f841bc23640 1 -- start start 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f841bc23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101750 0x7f8414197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f841bc23640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414102940 0x7f8414197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f841bc23640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8414103300 0x7f841419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f841bc23640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f841419f310 con 0x7f8414101750 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f841bc23640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f841419f480 con 0x7f8414103300 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f841bc23640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f841419f640 con 0x7f8414102940 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f84199bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101750 0x7f8414197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f84199bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101750 0x7f8414197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33746/0 (socket says 172.21.15.7:33746) 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f84199bf640 1 -- 172.21.15.7:0/4090573075 learned_addr learned my addr 172.21.15.7:0/4090573075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f84191be640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414102940 0x7f8414197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f841a1c0640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8414103300 0x7f841419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f84199bf640 1 -- 172.21.15.7:0/4090573075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414102940 msgr2=0x7f8414197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f84199bf640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414102940 0x7f8414197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f84199bf640 1 -- 172.21.15.7:0/4090573075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8414103300 msgr2=0x7f841419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f84199bf640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8414103300 0x7f841419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f84199bf640 1 -- 172.21.15.7:0/4090573075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8408009560 con 0x7f8414101750 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f84199bf640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101750 0x7f8414197820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8408000c00 tx=0x7f8408005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f8406ffd640 1 -- 172.21.15.7:0/4090573075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080043d0 con 0x7f8414101750 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f8406ffd640 1 -- 172.21.15.7:0/4090573075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84080052d0 con 0x7f8414101750 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f841bc23640 1 -- 172.21.15.7:0/4090573075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841419f8c0 con 0x7f8414101750 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f8406ffd640 1 -- 172.21.15.7:0/4090573075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408003d30 con 0x7f8414101750 2024-03-23T13:19:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.781+0000 7f841bc23640 1 -- 172.21.15.7:0/4090573075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841419fe20 con 0x7f8414101750 2024-03-23T13:19:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.783+0000 7f841bc23640 1 -- 172.21.15.7:0/4090573075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8414108ac0 con 0x7f8414101750 2024-03-23T13:19:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.783+0000 7f8406ffd640 1 -- 172.21.15.7:0/4090573075 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f8408005440 con 0x7f8414101750 2024-03-23T13:19:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.784+0000 7f8406ffd640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84000a21d0 0x7f84000a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.784+0000 7f84191be640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84000a21d0 0x7f84000a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.784+0000 7f8406ffd640 1 -- 172.21.15.7:0/4090573075 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f84080f0580 con 0x7f8414101750 2024-03-23T13:19:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.784+0000 7f84191be640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84000a21d0 0x7f84000a4690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8414198d30 tx=0x7f8410005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:08.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.788+0000 7f8406ffd640 1 -- 172.21.15.7:0/4090573075 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f84080b9630 con 0x7f8414101750 2024-03-23T13:19:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.973+0000 7f841bc23640 1 -- 172.21.15.7:0/4090573075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8414108530 con 0x7f8414101750 2024-03-23T13:19:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.975+0000 7f8406ffd640 1 -- 172.21.15.7:0/4090573075 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22558 (secure 0 0 0) 0x7f8408043090 con 0x7f8414101750 2024-03-23T13:19:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:08.975 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:18:43.667397+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:19:08.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 -- 172.21.15.7:0/4090573075 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84000a21d0 msgr2=0x7f84000a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84000a21d0 0x7f84000a4690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8414198d30 tx=0x7f8410005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 -- 172.21.15.7:0/4090573075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101750 msgr2=0x7f8414197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101750 0x7f8414197820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8408000c00 tx=0x7f8408005e80 comp rx=0 tx=0).stop 2024-03-23T13:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 -- 172.21.15.7:0/4090573075 shutdown_connections 2024-03-23T13:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f84000a21d0 0x7f84000a4690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8414103300 0x7f841419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8414102940 0x7f8414197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 --2- 172.21.15.7:0/4090573075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8414101750 0x7f8414197820 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 -- 172.21.15.7:0/4090573075 >> 172.21.15.7:0/4090573075 conn(0x7f84140fccc0 msgr2=0x7f8414105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 -- 172.21.15.7:0/4090573075 shutdown_connections 2024-03-23T13:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:08.977+0000 7f841bc23640 1 -- 172.21.15.7:0/4090573075 wait complete. 2024-03-23T13:19:08.987 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:08 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4291833758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:08.987 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:08 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1548993275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:08.987 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:08 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/131084074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:08.989 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:19:09.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:08 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/4291833758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:09.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:08 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1548993275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:09.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:08 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/131084074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:09.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.090+0000 7ff263dc7640 1 -- 172.21.15.7:0/679445274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff25c102260 msgr2=0x7ff25c108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.090+0000 7ff263dc7640 1 --2- 172.21.15.7:0/679445274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff25c102260 0x7ff25c108af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff254009ca0 tx=0x7ff25402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.090+0000 7ff263dc7640 1 -- 172.21.15.7:0/679445274 shutdown_connections 2024-03-23T13:19:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.090+0000 7ff263dc7640 1 --2- 172.21.15.7:0/679445274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff25c102260 0x7ff25c108af0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.090+0000 7ff263dc7640 1 --2- 172.21.15.7:0/679445274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25c1018a0 0x7ff25c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.090+0000 7ff263dc7640 1 --2- 172.21.15.7:0/679445274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1006b0 0x7ff25c100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.090+0000 7ff263dc7640 1 -- 172.21.15.7:0/679445274 >> 172.21.15.7:0/679445274 conn(0x7ff25c0fbc40 msgr2=0x7ff25c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.090+0000 7ff263dc7640 1 -- 172.21.15.7:0/679445274 shutdown_connections 2024-03-23T13:19:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.090+0000 7ff263dc7640 1 -- 172.21.15.7:0/679445274 wait complete. 2024-03-23T13:19:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.090+0000 7ff263dc7640 1 Processor -- start 2024-03-23T13:19:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff263dc7640 1 -- start start 2024-03-23T13:19:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff263dc7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1006b0 0x7ff25c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff263dc7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff25c1018a0 0x7ff25c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff263dc7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25c102260 0x7ff25c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff263dc7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c19f230 con 0x7ff25c1018a0 2024-03-23T13:19:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff263dc7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c19f3a0 con 0x7ff25c102260 2024-03-23T13:19:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff263dc7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c19f560 con 0x7ff25c1006b0 2024-03-23T13:19:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff261362640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff25c1018a0 0x7ff25c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff261362640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff25c1018a0 0x7ff25c197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33766/0 (socket says 172.21.15.7:33766) 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff261362640 1 -- 172.21.15.7:0/4048920169 learned_addr learned my addr 172.21.15.7:0/4048920169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff261b63640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1006b0 0x7ff25c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff262364640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25c102260 0x7ff25c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff261362640 1 -- 172.21.15.7:0/4048920169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1006b0 msgr2=0x7ff25c197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff261362640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1006b0 0x7ff25c197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff261362640 1 -- 172.21.15.7:0/4048920169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25c102260 msgr2=0x7ff25c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff261362640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25c102260 0x7ff25c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff261362640 1 -- 172.21.15.7:0/4048920169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff254009900 con 0x7ff25c1018a0 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff261362640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff25c1018a0 0x7ff25c197c80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00e9f0 tx=0x7ff24c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff252ffd640 1 -- 172.21.15.7:0/4048920169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c009700 con 0x7ff25c1018a0 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff263dc7640 1 -- 172.21.15.7:0/4048920169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c19f840 con 0x7ff25c1018a0 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff252ffd640 1 -- 172.21.15.7:0/4048920169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff24c00cd80 con 0x7ff25c1018a0 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.091+0000 7ff252ffd640 1 -- 172.21.15.7:0/4048920169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c018920 con 0x7ff25c1018a0 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.092+0000 7ff263dc7640 1 -- 172.21.15.7:0/4048920169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c19fda0 con 0x7ff25c1018a0 2024-03-23T13:19:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.093+0000 7ff263dc7640 1 -- 172.21.15.7:0/4048920169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25c1079d0 con 0x7ff25c1018a0 2024-03-23T13:19:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.094+0000 7ff252ffd640 1 -- 172.21.15.7:0/4048920169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7ff24c018a80 con 0x7ff25c1018a0 2024-03-23T13:19:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.094+0000 7ff252ffd640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff2480a2150 0x7ff2480a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.095+0000 7ff261b63640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff2480a2150 0x7ff2480a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.095+0000 7ff252ffd640 1 -- 172.21.15.7:0/4048920169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff24c014070 con 0x7ff25c1018a0 2024-03-23T13:19:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.095+0000 7ff261b63640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff2480a2150 0x7ff2480a4610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff25c101700 tx=0x7ff258005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.099+0000 7ff252ffd640 1 -- 172.21.15.7:0/4048920169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff24c053280 con 0x7ff25c1018a0 2024-03-23T13:19:09.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:08 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/4291833758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:09.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:08 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1548993275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:09.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:08 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/131084074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.323+0000 7ff263dc7640 1 -- 172.21.15.7:0/4048920169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff25c107440 con 0x7ff25c1018a0 2024-03-23T13:19:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.324+0000 7ff252ffd640 1 -- 172.21.15.7:0/4048920169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11917 (secure 0 0 0) 0x7ff24c0970c0 con 0x7ff25c1018a0 2024-03-23T13:19:09.324 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:09.325 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T13:17:40.597309+0000","modified":"2024-03-23T13:17:48.259350+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24475,"mds_3":24470,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6841/3415570481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3415570481},{"type":"v1","addr":"172.21.15.7:6841","nonce":3415570481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/791500055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":791500055},{"type":"v1","addr":"172.21.15.130:6839","nonce":791500055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24470,24463,24475,24503,24496]},"id":1}]} 2024-03-23T13:19:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-23T13:19:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 -- 172.21.15.7:0/4048920169 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff2480a2150 msgr2=0x7ff2480a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff2480a2150 0x7ff2480a4610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff25c101700 tx=0x7ff258005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 -- 172.21.15.7:0/4048920169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff25c1018a0 msgr2=0x7ff25c197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff25c1018a0 0x7ff25c197c80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00e9f0 tx=0x7ff24c00eeb0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 -- 172.21.15.7:0/4048920169 shutdown_connections 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff2480a2150 0x7ff2480a4610 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25c102260 0x7ff25c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff25c1018a0 0x7ff25c197c80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 --2- 172.21.15.7:0/4048920169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1006b0 0x7ff25c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 -- 172.21.15.7:0/4048920169 >> 172.21.15.7:0/4048920169 conn(0x7ff25c0fbc40 msgr2=0x7ff25c104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 -- 172.21.15.7:0/4048920169 shutdown_connections 2024-03-23T13:19:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.326+0000 7ff263dc7640 1 -- 172.21.15.7:0/4048920169 wait complete. 2024-03-23T13:19:09.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-23T13:17:40.597309+0000', 'modified': '2024-03-23T13:17:48.259350+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24503, 'mds_1': 24496, 'mds_2': 24475, 'mds_3': 24470, 'mds_4': 24463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.7:6839/388542752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6838', 'nonce': 388542752}, {'type': 'v1', 'addr': '172.21.15.7:6839', 'nonce': 388542752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14649': {'gid': 14649, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.7:6841/3415570481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6840', 'nonce': 3415570481}, {'type': 'v1', 'addr': '172.21.15.7:6841', 'nonce': 3415570481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6833/3818242586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6832', 'nonce': 3818242586}, {'type': 'v1', 'addr': '172.21.15.130:6833', 'nonce': 3818242586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6833/1319723529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 1319723529}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 1319723529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6835/2599360099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6834', 'nonce': 2599360099}, {'type': 'v1', 'addr': '172.21.15.130:6835', 'nonce': 2599360099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.155:6835/1880391142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 1880391142}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 1880391142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.130:6837/877142007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 877142007}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 877142007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.155:6837/3999767587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 3999767587}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 3999767587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.155:6839/131946068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 131946068}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 131946068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.130:6839/791500055', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 791500055}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 791500055}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24503, 'qdb_cluster': [24470, 24463, 24475, 24503, 24496]} 2024-03-23T13:19:09.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-23T13:19:09.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-23T13:19:09.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.441+0000 7f04af288640 1 -- 172.21.15.7:0/586955970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a8107410 msgr2=0x7f04a8109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.441+0000 7f04af288640 1 --2- 172.21.15.7:0/586955970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a8107410 0x7f04a8109860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009be0 tx=0x7f04a002bef0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.441+0000 7f04af288640 1 -- 172.21.15.7:0/586955970 shutdown_connections 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.441+0000 7f04af288640 1 --2- 172.21.15.7:0/586955970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a8107410 0x7f04a8109860 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.441+0000 7f04af288640 1 --2- 172.21.15.7:0/586955970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a8104ae0 0x7f04a8106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.441+0000 7f04af288640 1 --2- 172.21.15.7:0/586955970 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04a81021b0 0x7f04a81045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.441+0000 7f04af288640 1 -- 172.21.15.7:0/586955970 >> 172.21.15.7:0/586955970 conn(0x7f04a80fbbc0 msgr2=0x7f04a80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.441+0000 7f04af288640 1 -- 172.21.15.7:0/586955970 shutdown_connections 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.441+0000 7f04af288640 1 -- 172.21.15.7:0/586955970 wait complete. 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04af288640 1 Processor -- start 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04af288640 1 -- start start 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04af288640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a81021b0 0x7f04a8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04af288640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a8104ae0 0x7f04a8197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04af288640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04a8107410 0x7f04a819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04af288640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04a819f260 con 0x7f04a8104ae0 2024-03-23T13:19:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04af288640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f04a819f3d0 con 0x7f04a8107410 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04af288640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04a819f590 con 0x7f04a81021b0 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04ac823640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a8104ae0 0x7f04a8197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04ac823640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a8104ae0 0x7f04a8197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33784/0 (socket says 172.21.15.7:33784) 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04ac823640 1 -- 172.21.15.7:0/628174023 learned_addr learned my addr 172.21.15.7:0/628174023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04ad024640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a81021b0 0x7f04a8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04ad825640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04a8107410 0x7f04a819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04ac823640 1 -- 172.21.15.7:0/628174023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a81021b0 msgr2=0x7f04a8197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04ac823640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a81021b0 0x7f04a8197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04ac823640 1 -- 172.21.15.7:0/628174023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04a8107410 msgr2=0x7f04a819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04ac823640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04a8107410 0x7f04a819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.442+0000 7f04ac823640 1 -- 172.21.15.7:0/628174023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0498009560 con 0x7f04a8104ae0 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.443+0000 7f04ac823640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a8104ae0 0x7f04a8197cb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f049800e9a0 tx=0x7f049800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.443+0000 7f049e7fc640 1 -- 172.21.15.7:0/628174023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049800cd80 con 0x7f04a8104ae0 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.443+0000 7f049e7fc640 1 -- 172.21.15.7:0/628174023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04980035f0 con 0x7f04a8104ae0 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.443+0000 7f04af288640 1 -- 172.21.15.7:0/628174023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a0009840 con 0x7f04a8104ae0 2024-03-23T13:19:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.443+0000 7f049e7fc640 1 -- 172.21.15.7:0/628174023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498010690 con 0x7f04a8104ae0 2024-03-23T13:19:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.443+0000 7f04af288640 1 -- 172.21.15.7:0/628174023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a819fbf0 con 0x7f04a8104ae0 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.445+0000 7f049e7fc640 1 -- 172.21.15.7:0/628174023 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f0498003ad0 con 0x7f04a8104ae0 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.445+0000 7f04b1e6b740 1 Processor -- start 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.445+0000 7f04b1e6b740 1 -- start start 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.445+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e52a560 0x55964e52a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.445+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e52fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.445+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e5300c0 0x55964e532550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55964e499180 con 0x55964e52af50 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55964e4966b0 con 0x55964e52a560 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55964e491e90 con 0x55964e5300c0 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f049e7fc640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f04940a2150 0x7f04940a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad024640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e52fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad024640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e52fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33788/0 (socket says 172.21.15.7:33788) 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad024640 1 -- 172.21.15.7:0/3994869763 learned_addr learned my addr 172.21.15.7:0/3994869763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad825640 1 --2- 172.21.15.7:0/3994869763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e52a560 0x55964e52a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ac823640 1 --2- 172.21.15.7:0/3994869763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e5300c0 0x55964e532550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad825640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f04940a2150 0x7f04940a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f049e7fc640 1 -- 172.21.15.7:0/628174023 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0498014070 con 0x7f04a8104ae0 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad024640 1 -- 172.21.15.7:0/3994869763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e5300c0 msgr2=0x55964e532550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad024640 1 --2- 172.21.15.7:0/3994869763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e5300c0 0x55964e532550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad024640 1 -- 172.21.15.7:0/3994869763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e52a560 msgr2=0x55964e52a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad024640 1 --2- 172.21.15.7:0/3994869763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e52a560 0x55964e52a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad024640 1 -- 172.21.15.7:0/3994869763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964e44ed80 con 0x55964e52af50 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad825640 1 --2- 172.21.15.7:0/3994869763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e52a560 0x55964e52a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad825640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f04940a2150 0x7f04940a4610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f04a0004af0 tx=0x7f04a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04ad024640 1 --2- 172.21.15.7:0/3994869763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e52fa50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009d20 tx=0x7f04a4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3695e5057b86bf61 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04937fe640 1 -- 172.21.15.7:0/3994869763 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4019630 con 0x55964e52af50 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04937fe640 1 -- 172.21.15.7:0/3994869763 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04a4019c70 con 0x55964e52af50 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.446+0000 7f04937fe640 1 -- 172.21.15.7:0/3994869763 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a400ec40 con 0x55964e52af50 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.447+0000 7f04b1e6b740 1 -- 172.21.15.7:0/3994869763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 msgr2=0x55964e52fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.447+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/3994869763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e52fa50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009d20 tx=0x7f04a4011810 comp rx=0 tx=0).stop 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.447+0000 7f04b1e6b740 1 -- 172.21.15.7:0/3994869763 shutdown_connections 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.447+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/3994869763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e5300c0 0x55964e532550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.447+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/3994869763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e52fa50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.447+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/3994869763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e52a560 0x55964e52a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.447+0000 7f04b1e6b740 1 -- 172.21.15.7:0/3994869763 >> 172.21.15.7:0/3994869763 conn(0x55964e525ad0 msgr2=0x55964e525ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.447+0000 7f04b1e6b740 1 -- 172.21.15.7:0/3994869763 shutdown_connections 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.447+0000 7f04b1e6b740 1 -- 172.21.15.7:0/3994869763 wait complete. 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.447+0000 7f04b1e6b740 1 -- start start 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04b1e6b740 1 Processor -- start 2024-03-23T13:19:09.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52a560 0x55964e534540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52af50 0x55964e534a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e5300c0 0x55964e53bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55964e436d50 con 0x55964e52a560 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55964e165310 con 0x55964e5300c0 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55964e4178f0 con 0x55964e52af50 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad024640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52a560 0x55964e534540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04b1e6b740 10 client.0 _ll_register_callbacks cb 0x55964e1a2ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad024640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52a560 0x55964e534540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33800/0 (socket says 172.21.15.7:33800) 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad024640 1 -- 172.21.15.7:0/695458801 learned_addr learned my addr 172.21.15.7:0/695458801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ac823640 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52af50 0x55964e534a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad825640 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e5300c0 0x55964e53bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad024640 1 -- 172.21.15.7:0/695458801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52af50 msgr2=0x55964e534a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad024640 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52af50 0x55964e534a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad024640 1 -- 172.21.15.7:0/695458801 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e5300c0 msgr2=0x55964e53bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad024640 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e5300c0 0x55964e53bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad024640 1 -- 172.21.15.7:0/695458801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a4009950 con 0x55964e52a560 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad825640 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e5300c0 0x55964e53bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f04ad024640 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52a560 0x55964e534540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009cf0 tx=0x7f04a4004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335949c2888e91b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.448+0000 7f0490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f0490ff9640 1 -- 172.21.15.7:0/695458801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a400e5a0 con 0x55964e52a560 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f0490ff9640 1 -- 172.21.15.7:0/695458801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f04a400e3b0 con 0x55964e52a560 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f04b1e6b740 1 -- 172.21.15.7:0/695458801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55964e1916c0 con 0x55964e52a560 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f0490ff9640 1 -- 172.21.15.7:0/695458801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04a4009ed0 con 0x55964e52a560 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f04b1e6b740 10 client.15069 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f04b1e6b740 1 -- 172.21.15.7:0/695458801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55964e490510 con 0x55964e52a560 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f0490ff9640 1 -- 172.21.15.7:0/695458801 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04a4034020 con 0x55964e52a560 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f0490ff9640 1 -- 172.21.15.7:0/695458801 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f04a4003880 con 0x55964e52a560 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f04b1e6b740 10 client.15069 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f04b1e6b740 10 client.15069 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f04b1e6b740 20 client.15069 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55964e540c70 0x55964e5420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f04b1e6b740 4 client.15069 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f04b1e6b740 1 -- 172.21.15.7:0/695458801 --> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55964e1e2270 con 0x55964e540c70 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.449+0000 7f04ac823640 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55964e540c70 0x55964e5420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.450+0000 7f04ac823640 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55964e540c70 0x55964e5420e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f63f8187ada7d80a server_cookie=701d257ad29c81cd in_seq=0 out_seq=0 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.450+0000 7f0490ff9640 10 client.15069 ms_handle_connect on v2:172.21.15.155:6834/1880391142 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.450+0000 7f0490ff9640 1 -- 172.21.15.7:0/695458801 <== mds.4 v2:172.21.15.155:6834/1880391142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55964e1e2270 con 0x55964e540c70 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f0490ff9640 10 client.15069 handle_command_reply: tid=0 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 client.15069 shutdown 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 -- 172.21.15.7:0/695458801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52a560 msgr2=0x55964e534540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52a560 0x55964e534540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009cf0 tx=0x7f04a4004490 comp rx=0 tx=0).stop 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 -- 172.21.15.7:0/695458801 shutdown_connections 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e5300c0 0x55964e53bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52af50 0x55964e534a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55964e540c70 0x55964e5420e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/695458801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52a560 0x55964e534540 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 -- 172.21.15.7:0/695458801 >> 172.21.15.7:0/695458801 conn(0x55964e525ad0 msgr2=0x55964e524ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f0490ff9640 0 client.15069 ms_handle_reset on v2:172.21.15.155:6834/1880391142 2024-03-23T13:19:09.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 -- 172.21.15.7:0/695458801 shutdown_connections 2024-03-23T13:19:09.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 1 -- 172.21.15.7:0/695458801 wait complete. 2024-03-23T13:19:09.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.451+0000 7f04b1e6b740 20 client.15069 trim_cache size 0 max 16384 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.469+0000 7f04b1e6b740 1 Processor -- start 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.469+0000 7f04b1e6b740 1 -- start start 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.469+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e525ad0 0x55964e533290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.469+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 0x55964e53ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.469+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e525eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.469+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55964e1e2270 con 0x55964e52af50 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.469+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55964e53c5b0 con 0x55964e525ad0 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.469+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55964e44af70 con 0x55964e52a560 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.469+0000 7f04ad825640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e525eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ad825640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e525eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33808/0 (socket says 172.21.15.7:33808) 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ad825640 1 -- 172.21.15.7:0/4014386884 learned_addr learned my addr 172.21.15.7:0/4014386884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ac823640 1 --2- 172.21.15.7:0/4014386884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 0x55964e53ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ad024640 1 --2- 172.21.15.7:0/4014386884 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e525ad0 0x55964e533290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ad825640 1 -- 172.21.15.7:0/4014386884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 msgr2=0x55964e53ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ad825640 1 --2- 172.21.15.7:0/4014386884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 0x55964e53ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ad825640 1 -- 172.21.15.7:0/4014386884 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e525ad0 msgr2=0x55964e533290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ad825640 1 --2- 172.21.15.7:0/4014386884 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e525ad0 0x55964e533290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ad825640 1 -- 172.21.15.7:0/4014386884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964e42ca80 con 0x55964e52af50 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ac823640 1 --2- 172.21.15.7:0/4014386884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 0x55964e53ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f04ad825640 1 --2- 172.21.15.7:0/4014386884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e525eb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f04a0035000 tx=0x7f04a0043070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd62c1cac7db4d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f0492ffd640 1 -- 172.21.15.7:0/4014386884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0003b70 con 0x55964e52af50 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f0492ffd640 1 -- 172.21.15.7:0/4014386884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04a0047040 con 0x55964e52af50 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.470+0000 7f0492ffd640 1 -- 172.21.15.7:0/4014386884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a00347b0 con 0x55964e52af50 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 -- 172.21.15.7:0/4014386884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 msgr2=0x55964e525eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/4014386884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e525eb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f04a0035000 tx=0x7f04a0043070 comp rx=0 tx=0).stop 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 -- 172.21.15.7:0/4014386884 shutdown_connections 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/4014386884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e525eb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/4014386884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 0x55964e53ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/4014386884 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e525ad0 0x55964e533290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 -- 172.21.15.7:0/4014386884 >> 172.21.15.7:0/4014386884 conn(0x55964e540040 msgr2=0x55964e52f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 -- 172.21.15.7:0/4014386884 shutdown_connections 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 -- 172.21.15.7:0/4014386884 wait complete. 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 -- start start 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 Processor -- start 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e525ad0 0x55964e5399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 0x55964e5a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e5a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55964e5a55f0 con 0x55964e52af50 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55964e5a5760 con 0x55964e525ad0 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55964e5a5920 con 0x55964e52a560 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04b1e6b740 10 client.0 _ll_register_callbacks cb 0x7f04a401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.471+0000 7f04ad825640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e5a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ad825640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e5a4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33814/0 (socket says 172.21.15.7:33814) 2024-03-23T13:19:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ad825640 1 -- 172.21.15.7:0/1885306377 learned_addr learned my addr 172.21.15.7:0/1885306377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ac823640 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 0x55964e5a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ad024640 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e525ad0 0x55964e5399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ad825640 1 -- 172.21.15.7:0/1885306377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 msgr2=0x55964e5a29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ad825640 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 0x55964e5a29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ad825640 1 -- 172.21.15.7:0/1885306377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e525ad0 msgr2=0x55964e5399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ad825640 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e525ad0 0x55964e5399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ad825640 1 -- 172.21.15.7:0/1885306377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a819fbf0 con 0x55964e52af50 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ac823640 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 0x55964e5a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04ad825640 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e5a4f10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04a00049d0 tx=0x7f04a0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a924468574956dc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f0490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f0490ff9640 1 -- 172.21.15.7:0/1885306377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0034940 con 0x55964e52af50 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f0490ff9640 1 -- 172.21.15.7:0/1885306377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f04a005e070 con 0x55964e52af50 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04b1e6b740 1 -- 172.21.15.7:0/1885306377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a4019c70 con 0x55964e52af50 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f0490ff9640 1 -- 172.21.15.7:0/1885306377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04a0069030 con 0x55964e52af50 2024-03-23T13:19:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04b1e6b740 10 client.15075 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f04b1e6b740 1 -- 172.21.15.7:0/1885306377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x7f04a0030070 con 0x55964e52af50 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.472+0000 7f0490ff9640 1 -- 172.21.15.7:0/1885306377 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04a4019c70 con 0x55964e52af50 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f0490ff9640 1 -- 172.21.15.7:0/1885306377 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f04a004c050 con 0x55964e52af50 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f04b1e6b740 10 client.15075 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f04b1e6b740 10 client.15075 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f04b1e6b740 20 client.15075 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55964e5300c0 0x55964e5a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f04b1e6b740 4 client.15075 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f04b1e6b740 1 -- 172.21.15.7:0/1885306377 --> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] -- command(tid 0: {"prefix": "status"}) -- 0x55964e1ce610 con 0x55964e5300c0 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f04ad024640 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55964e5300c0 0x55964e5a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f0490ff9640 10 client.15075 ms_handle_connect on v2:172.21.15.155:6834/1880391142 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f04ad024640 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55964e5300c0 0x55964e5a9c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d6ce1eda1161e5d7 server_cookie=8f52568c901948ab in_seq=0 out_seq=0 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f0490ff9640 1 -- 172.21.15.7:0/1885306377 <== mds.4 v2:172.21.15.155:6834/1880391142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55964e1ce610 con 0x55964e5300c0 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f0490ff9640 10 client.15075 handle_command_reply: tid=0 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.473+0000 7f04b1e6b740 1 client.15075 shutdown 2024-03-23T13:19:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 1 -- 172.21.15.7:0/1885306377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 msgr2=0x55964e5a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e5a4f10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04a00049d0 tx=0x7f04a0003c20 comp rx=0 tx=0).stop 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 1 -- 172.21.15.7:0/1885306377 shutdown_connections 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55964e5300c0 0x55964e5a9c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55964e52af50 0x55964e5a4f10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55964e52a560 0x55964e5a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 1 --2- 172.21.15.7:0/1885306377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55964e525ad0 0x55964e5399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 1 -- 172.21.15.7:0/1885306377 >> 172.21.15.7:0/1885306377 conn(0x55964e540040 msgr2=0x55964e52f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f0490ff9640 0 client.15075 ms_handle_reset on v2:172.21.15.155:6834/1880391142 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 1 -- 172.21.15.7:0/1885306377 shutdown_connections 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 1 -- 172.21.15.7:0/1885306377 wait complete. 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04b1e6b740 20 client.15075 trim_cache size 0 max 16384 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:19:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 4, 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24463, 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 81.901629530999998, 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 109.55565068200001 2024-03-23T13:19:09.481 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:19:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 -- 172.21.15.7:0/628174023 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f04940a2150 msgr2=0x7f04940a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f04940a2150 0x7f04940a4610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f04a0004af0 tx=0x7f04a0005fd0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 -- 172.21.15.7:0/628174023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a8104ae0 msgr2=0x7f04a8197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a8104ae0 0x7f04a8197cb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f049800e9a0 tx=0x7f049800ee60 comp rx=0 tx=0).stop 2024-03-23T13:19:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 -- 172.21.15.7:0/628174023 shutdown_connections 2024-03-23T13:19:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f04940a2150 0x7f04940a4610 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04a8107410 0x7f04a819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a8104ae0 0x7f04a8197cb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 --2- 172.21.15.7:0/628174023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a81021b0 0x7f04a8197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 -- 172.21.15.7:0/628174023 >> 172.21.15.7:0/628174023 conn(0x7f04a80fbbc0 msgr2=0x7f04a8102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 -- 172.21.15.7:0/628174023 shutdown_connections 2024-03-23T13:19:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.474+0000 7f04af288640 1 -- 172.21.15.7:0/628174023 wait complete. 2024-03-23T13:19:09.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-23T13:19:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 -- 172.21.15.7:0/1226121579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c1021d0 msgr2=0x7f372c108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 --2- 172.21.15.7:0/1226121579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c1021d0 0x7f372c108a60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3724009ca0 tx=0x7f372402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 -- 172.21.15.7:0/1226121579 shutdown_connections 2024-03-23T13:19:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 --2- 172.21.15.7:0/1226121579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c1021d0 0x7f372c108a60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 --2- 172.21.15.7:0/1226121579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c101810 0x7f372c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 --2- 172.21.15.7:0/1226121579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f372c100620 0x7f372c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 -- 172.21.15.7:0/1226121579 >> 172.21.15.7:0/1226121579 conn(0x7f372c0fbbb0 msgr2=0x7f372c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 -- 172.21.15.7:0/1226121579 shutdown_connections 2024-03-23T13:19:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 -- 172.21.15.7:0/1226121579 wait complete. 2024-03-23T13:19:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 Processor -- start 2024-03-23T13:19:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.589+0000 7f3734047640 1 -- start start 2024-03-23T13:19:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3734047640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f372c100620 0x7f372c10aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3734047640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c101810 0x7f372c109520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3734047640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c1021d0 0x7f372c109a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3734047640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f372c10a1c0 con 0x7f372c1021d0 2024-03-23T13:19:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3734047640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f372c10a330 con 0x7f372c100620 2024-03-23T13:19:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3734047640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1a1820 con 0x7f372c101810 2024-03-23T13:19:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f37325e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c1021d0 0x7f372c109a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f37325e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c1021d0 0x7f372c109a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33824/0 (socket says 172.21.15.7:33824) 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f37325e4640 1 -- 172.21.15.7:0/2565222198 learned_addr learned my addr 172.21.15.7:0/2565222198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3731de3640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f372c100620 0x7f372c10aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f37315e2640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c101810 0x7f372c109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f37325e4640 1 -- 172.21.15.7:0/2565222198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c101810 msgr2=0x7f372c109520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f37325e4640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c101810 0x7f372c109520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f37325e4640 1 -- 172.21.15.7:0/2565222198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f372c100620 msgr2=0x7f372c10aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f37325e4640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f372c100620 0x7f372c10aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f37325e4640 1 -- 172.21.15.7:0/2565222198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3724009900 con 0x7f372c1021d0 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f37325e4640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c1021d0 0x7f372c109a60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3724033000 tx=0x7f3724009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3722ffd640 1 -- 172.21.15.7:0/2565222198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37240329b0 con 0x7f372c1021d0 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3722ffd640 1 -- 172.21.15.7:0/2565222198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3724003e50 con 0x7f372c1021d0 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3734047640 1 -- 172.21.15.7:0/2565222198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c1a1a70 con 0x7f372c1021d0 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3722ffd640 1 -- 172.21.15.7:0/2565222198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724040450 con 0x7f372c1021d0 2024-03-23T13:19:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.590+0000 7f3734047640 1 -- 172.21.15.7:0/2565222198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1a1f20 con 0x7f372c1021d0 2024-03-23T13:19:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.592+0000 7f3722ffd640 1 -- 172.21.15.7:0/2565222198 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f372400a040 con 0x7f372c1021d0 2024-03-23T13:19:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3736c24740 1 Processor -- start 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3736c24740 1 -- start start 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3736c24740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad12980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3736c24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad17a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3736c24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad180c0 0x5605cad1a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3736c24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5605cac81180 con 0x5605cad12f50 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3736c24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5605cac7e6b0 con 0x5605cad180c0 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3736c24740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5605cac79e90 con 0x5605cad12560 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3722ffd640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f37180a2150 0x7f37180a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3731de3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad17a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f37315e2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad12980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3731de3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad17a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33828/0 (socket says 172.21.15.7:33828) 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3731de3640 1 -- 172.21.15.7:0/2065664891 learned_addr learned my addr 172.21.15.7:0/2065664891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f37315e2640 1 --2- 172.21.15.7:0/2065664891 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad180c0 0x5605cad1a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f37325e4640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f37180a2150 0x7f37180a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f3722ffd640 1 -- 172.21.15.7:0/2565222198 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f37240f0a60 con 0x7f372c1021d0 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f37315e2640 1 -- 172.21.15.7:0/2065664891 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad180c0 msgr2=0x5605cad1a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f37315e2640 1 --2- 172.21.15.7:0/2065664891 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad180c0 0x5605cad1a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f37315e2640 1 -- 172.21.15.7:0/2065664891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 msgr2=0x5605cad17a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f37315e2640 1 --2- 172.21.15.7:0/2065664891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad17a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.593+0000 7f37315e2640 1 -- 172.21.15.7:0/2065664891 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605cac36d80 con 0x5605cad12560 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.594+0000 7f37315e2640 1 --2- 172.21.15.7:0/2065664891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad12980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f372c10ac50 tx=0x7f371c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f756174c8cec565 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.594+0000 7f37325e4640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f37180a2150 0x7f37180a4610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f37240b9bb0 tx=0x7f372408c940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.594+0000 7f3717fff640 1 -- 172.21.15.7:0/2065664891 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c004050 con 0x5605cad12560 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.594+0000 7f3717fff640 1 -- 172.21.15.7:0/2065664891 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c00fca0 con 0x5605cad12560 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.594+0000 7f3717fff640 1 -- 172.21.15.7:0/2065664891 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0038f0 con 0x5605cad12560 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 -- 172.21.15.7:0/2065664891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 msgr2=0x5605cad12980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 --2- 172.21.15.7:0/2065664891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad12980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f372c10ac50 tx=0x7f371c004730 comp rx=0 tx=0).stop 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 -- 172.21.15.7:0/2065664891 shutdown_connections 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 --2- 172.21.15.7:0/2065664891 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad180c0 0x5605cad1a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 --2- 172.21.15.7:0/2065664891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad17a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 --2- 172.21.15.7:0/2065664891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad12980 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 -- 172.21.15.7:0/2065664891 >> 172.21.15.7:0/2065664891 conn(0x5605cad0dad0 msgr2=0x5605cad0ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 -- 172.21.15.7:0/2065664891 shutdown_connections 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 -- 172.21.15.7:0/2065664891 wait complete. 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 -- start start 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 Processor -- start 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad1c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12f50 0x5605cad1cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad23b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5605cac1ed50 con 0x5605cad180c0 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5605ca94d310 con 0x5605cad12f50 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.595+0000 7f3736c24740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5605cabff8f0 con 0x5605cad12560 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3736c24740 10 client.0 _ll_register_callbacks cb 0x5605ca98aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f37315e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad23b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f37315e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad23b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33830/0 (socket says 172.21.15.7:33830) 2024-03-23T13:19:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f37315e2640 1 -- 172.21.15.7:0/2671914685 learned_addr learned my addr 172.21.15.7:0/2671914685 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3731de3640 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12f50 0x5605cad1cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f37315e2640 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad1c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3731de3640 1 -- 172.21.15.7:0/2671914685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 msgr2=0x5605cad1c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3731de3640 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad1c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3731de3640 1 -- 172.21.15.7:0/2671914685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 msgr2=0x5605cad23b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3731de3640 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad23b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3731de3640 1 -- 172.21.15.7:0/2671914685 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371c019040 con 0x5605cad12f50 2024-03-23T13:19:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f37315e2640 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad23b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3731de3640 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12f50 0x5605cad1cae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5605cad13350 tx=0x7f3728012b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d91ac30b75e78a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f37157fa640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f37157fa640 1 -- 172.21.15.7:0/2671914685 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728017070 con 0x5605cad12f50 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3736c24740 1 -- 172.21.15.7:0/2671914685 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605ca9796c0 con 0x5605cad12f50 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f37157fa640 1 -- 172.21.15.7:0/2671914685 <== mon.2 v2:172.21.15.130:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3728004050 con 0x5605cad12f50 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f37157fa640 1 -- 172.21.15.7:0/2671914685 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3728022030 con 0x5605cad12f50 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3736c24740 10 client.24908 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.596+0000 7f3736c24740 1 -- 172.21.15.7:0/2671914685 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5605cac78510 con 0x5605cad12f50 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f37157fa640 1 -- 172.21.15.7:0/2671914685 <== mon.2 v2:172.21.15.130:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5605ca9796c0 con 0x5605cad12f50 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f37157fa640 1 -- 172.21.15.7:0/2671914685 <== mon.2 v2:172.21.15.130:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f37280038c0 con 0x5605cad12f50 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f3736c24740 10 client.24908 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f3736c24740 10 client.24908 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f3736c24740 20 client.24908 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f3736c24740 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x5605cad28e90 0x5605cad2a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f3736c24740 4 client.24908 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f3736c24740 1 -- 172.21.15.7:0/2671914685 --> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605ca9ca270 con 0x5605cad28e90 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f37315e2640 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x5605cad28e90 0x5605cad2a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f37315e2640 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x5605cad28e90 0x5605cad2a300 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6dc2923e4b15fa66 server_cookie=757a22bd7516c406 in_seq=0 out_seq=0 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.597+0000 7f37157fa640 10 client.24908 ms_handle_connect on v2:172.21.15.130:6836/877142007 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f37157fa640 1 -- 172.21.15.7:0/2671914685 <== mds.3 v2:172.21.15.130:6836/877142007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5605ca9ca270 con 0x5605cad28e90 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f37157fa640 10 client.24908 handle_command_reply: tid=0 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f3736c24740 1 client.24908 shutdown 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f3736c24740 1 -- 172.21.15.7:0/2671914685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12f50 msgr2=0x5605cad1cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f3736c24740 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12f50 0x5605cad1cae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5605cad13350 tx=0x7f3728012b10 comp rx=0 tx=0).stop 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f3736c24740 1 -- 172.21.15.7:0/2671914685 shutdown_connections 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f3736c24740 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad23b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f3736c24740 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12f50 0x5605cad1cae0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f3736c24740 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x5605cad28e90 0x5605cad2a300 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f3736c24740 1 --2- 172.21.15.7:0/2671914685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad1c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f37157fa640 0 client.24908 ms_handle_reset on v2:172.21.15.130:6836/877142007 2024-03-23T13:19:09.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f3736c24740 1 -- 172.21.15.7:0/2671914685 >> 172.21.15.7:0/2671914685 conn(0x5605cad0dad0 msgr2=0x5605cad0cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.598+0000 7f3736c24740 1 -- 172.21.15.7:0/2671914685 shutdown_connections 2024-03-23T13:19:09.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.599+0000 7f3736c24740 1 -- 172.21.15.7:0/2671914685 wait complete. 2024-03-23T13:19:09.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.599+0000 7f3736c24740 20 client.24908 trim_cache size 0 max 16384 2024-03-23T13:19:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f3736c24740 1 Processor -- start 2024-03-23T13:19:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f3736c24740 1 -- start start 2024-03-23T13:19:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f3736c24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12560 0x5605cad1a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f3736c24740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12f50 0x5605cad23ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f3736c24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad137b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f3736c24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5605cad29790 con 0x5605cad180c0 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f3736c24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5605cad289f0 con 0x5605cad12560 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f3736c24740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5605cad1ac10 con 0x5605cad12f50 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f37315e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f37315e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad137b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33842/0 (socket says 172.21.15.7:33842) 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f37315e2640 1 -- 172.21.15.7:0/723589851 learned_addr learned my addr 172.21.15.7:0/723589851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f3731de3640 1 --2- 172.21.15.7:0/723589851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12f50 0x5605cad23ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f37315e2640 1 --2- 172.21.15.7:0/723589851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12560 0x5605cad1a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f37315e2640 1 -- 172.21.15.7:0/723589851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12f50 msgr2=0x5605cad23ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f37315e2640 1 --2- 172.21.15.7:0/723589851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12f50 0x5605cad23ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f37315e2640 1 -- 172.21.15.7:0/723589851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12560 msgr2=0x5605cad1a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f37315e2640 1 --2- 172.21.15.7:0/723589851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12560 0x5605cad1a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f37315e2640 1 -- 172.21.15.7:0/723589851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605cac14a80 con 0x5605cad180c0 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f3731de3640 1 --2- 172.21.15.7:0/723589851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12f50 0x5605cad23ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.617+0000 7f37315e2640 1 --2- 172.21.15.7:0/723589851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad137b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f371c011710 tx=0x7f371c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70878289df33453 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f37177fe640 1 -- 172.21.15.7:0/723589851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c026040 con 0x5605cad180c0 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f37177fe640 1 -- 172.21.15.7:0/723589851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c004540 con 0x5605cad180c0 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f37177fe640 1 -- 172.21.15.7:0/723589851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0053f0 con 0x5605cad180c0 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f3736c24740 1 -- 172.21.15.7:0/723589851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 msgr2=0x5605cad137b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f3736c24740 1 --2- 172.21.15.7:0/723589851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad137b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f371c011710 tx=0x7f371c005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f3736c24740 1 -- 172.21.15.7:0/723589851 shutdown_connections 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f3736c24740 1 --2- 172.21.15.7:0/723589851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad180c0 0x5605cad137b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f3736c24740 1 --2- 172.21.15.7:0/723589851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12f50 0x5605cad23ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f3736c24740 1 --2- 172.21.15.7:0/723589851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad12560 0x5605cad1a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f3736c24740 1 -- 172.21.15.7:0/723589851 >> 172.21.15.7:0/723589851 conn(0x5605cad0dad0 msgr2=0x5605cad0cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f3736c24740 1 -- 172.21.15.7:0/723589851 shutdown_connections 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.618+0000 7f3736c24740 1 -- 172.21.15.7:0/723589851 wait complete. 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3736c24740 1 -- start start 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3736c24740 1 Processor -- start 2024-03-23T13:19:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3736c24740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad16310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3736c24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad16850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3736c24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad180c0 0x5605cad220f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3736c24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5605cad22630 con 0x5605cad12f50 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3736c24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5605cad227a0 con 0x5605cad180c0 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3736c24740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5605cad22960 con 0x5605cad12560 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3736c24740 10 client.0 _ll_register_callbacks cb 0x7f3728009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3731de3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad16850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3731de3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad16850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33844/0 (socket says 172.21.15.7:33844) 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3731de3640 1 -- 172.21.15.7:0/4178697953 learned_addr learned my addr 172.21.15.7:0/4178697953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f37315e2640 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad16310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f37315e2640 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad180c0 0x5605cad220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3731de3640 1 -- 172.21.15.7:0/4178697953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 msgr2=0x5605cad16310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3731de3640 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad16310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3731de3640 1 -- 172.21.15.7:0/4178697953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad180c0 msgr2=0x5605cad220f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3731de3640 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad180c0 0x5605cad220f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.619+0000 7f3731de3640 1 -- 172.21.15.7:0/4178697953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605cac78510 con 0x5605cad12f50 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f3731de3640 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad16850 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5605cad1da80 tx=0x7f372800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e8cfce6ba51425 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f37157fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f37157fa640 1 -- 172.21.15.7:0/4178697953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728022070 con 0x5605cad12f50 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f3736c24740 1 -- 172.21.15.7:0/4178697953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605ca9b6610 con 0x5605cad12f50 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f37157fa640 1 -- 172.21.15.7:0/4178697953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3728023070 con 0x5605cad12f50 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f37157fa640 1 -- 172.21.15.7:0/4178697953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3728017040 con 0x5605cad12f50 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f3736c24740 10 client.15093 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f3736c24740 1 -- 172.21.15.7:0/4178697953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x7f371c029040 con 0x5605cad12f50 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f37157fa640 1 -- 172.21.15.7:0/4178697953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5605ca9b6610 con 0x5605cad12f50 2024-03-23T13:19:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f37157fa640 1 -- 172.21.15.7:0/4178697953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f37280106d0 con 0x5605cad12f50 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f3736c24740 10 client.15093 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f3736c24740 10 client.15093 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f3736c24740 20 client.15093 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f3736c24740 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x5605cad92d60 0x5605cad91a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f3736c24740 4 client.15093 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f3736c24740 1 -- 172.21.15.7:0/4178697953 --> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] -- command(tid 0: {"prefix": "status"}) -- 0x5605ca9255e0 con 0x5605cad92d60 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.620+0000 7f37315e2640 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x5605cad92d60 0x5605cad91a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f37157fa640 10 client.15093 ms_handle_connect on v2:172.21.15.130:6836/877142007 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f37315e2640 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x5605cad92d60 0x5605cad91a70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69e28629062377b5 server_cookie=93e4a20cbf05e201 in_seq=0 out_seq=0 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f37157fa640 1 -- 172.21.15.7:0/4178697953 <== mds.3 v2:172.21.15.130:6836/877142007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5605ca9255e0 con 0x5605cad92d60 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f37157fa640 10 client.15093 handle_command_reply: tid=0 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 client.15093 shutdown 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 -- 172.21.15.7:0/4178697953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 msgr2=0x5605cad16850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad16850 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5605cad1da80 tx=0x7f372800f810 comp rx=0 tx=0).stop 2024-03-23T13:19:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 -- 172.21.15.7:0/4178697953 shutdown_connections 2024-03-23T13:19:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5605cad180c0 0x5605cad220f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605cad12f50 0x5605cad16850 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x5605cad92d60 0x5605cad91a70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 --2- 172.21.15.7:0/4178697953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605cad12560 0x5605cad16310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 -- 172.21.15.7:0/4178697953 >> 172.21.15.7:0/4178697953 conn(0x5605cad0dad0 msgr2=0x5605cad0cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f37157fa640 0 client.15093 ms_handle_reset on v2:172.21.15.130:6836/877142007 2024-03-23T13:19:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 -- 172.21.15.7:0/4178697953 shutdown_connections 2024-03-23T13:19:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 1 -- 172.21.15.7:0/4178697953 wait complete. 2024-03-23T13:19:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.621+0000 7f3736c24740 20 client.15093 trim_cache size 0 max 16384 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 3, 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24470, 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 83.055940559000007, 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T13:19:09.629 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 104.010087207 2024-03-23T13:19:09.630 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:19:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 -- 172.21.15.7:0/2565222198 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f37180a2150 msgr2=0x7f37180a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f37180a2150 0x7f37180a4610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f37240b9bb0 tx=0x7f372408c940 comp rx=0 tx=0).stop 2024-03-23T13:19:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 -- 172.21.15.7:0/2565222198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c1021d0 msgr2=0x7f372c109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c1021d0 0x7f372c109a60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3724033000 tx=0x7f3724009360 comp rx=0 tx=0).stop 2024-03-23T13:19:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 -- 172.21.15.7:0/2565222198 shutdown_connections 2024-03-23T13:19:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f37180a2150 0x7f37180a4610 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c1021d0 0x7f372c109a60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c101810 0x7f372c109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 --2- 172.21.15.7:0/2565222198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f372c100620 0x7f372c10aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 -- 172.21.15.7:0/2565222198 >> 172.21.15.7:0/2565222198 conn(0x7f372c0fbbb0 msgr2=0x7f372c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 -- 172.21.15.7:0/2565222198 shutdown_connections 2024-03-23T13:19:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.622+0000 7f3734047640 1 -- 172.21.15.7:0/2565222198 wait complete. 2024-03-23T13:19:09.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T13:19:09.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.736+0000 7f26a96b5640 1 -- 172.21.15.7:0/1653145268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a4100620 msgr2=0x7f26a4100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.736+0000 7f26a96b5640 1 --2- 172.21.15.7:0/1653145268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a4100620 0x7f26a4100a40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2694009900 tx=0x7f269402bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:09.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.736+0000 7f26a96b5640 1 -- 172.21.15.7:0/1653145268 shutdown_connections 2024-03-23T13:19:09.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.736+0000 7f26a96b5640 1 --2- 172.21.15.7:0/1653145268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26a41021d0 0x7f26a4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.736+0000 7f26a96b5640 1 --2- 172.21.15.7:0/1653145268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a4101810 0x7f26a4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.736+0000 7f26a96b5640 1 --2- 172.21.15.7:0/1653145268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a4100620 0x7f26a4100a40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.736+0000 7f26a96b5640 1 -- 172.21.15.7:0/1653145268 >> 172.21.15.7:0/1653145268 conn(0x7f26a40fbbb0 msgr2=0x7f26a40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.736+0000 7f26a96b5640 1 -- 172.21.15.7:0/1653145268 shutdown_connections 2024-03-23T13:19:09.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.736+0000 7f26a96b5640 1 -- 172.21.15.7:0/1653145268 wait complete. 2024-03-23T13:19:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.736+0000 7f26a96b5640 1 Processor -- start 2024-03-23T13:19:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f26a96b5640 1 -- start start 2024-03-23T13:19:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f26a96b5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a4100620 0x7f26a4196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f26a96b5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a4101810 0x7f26a4196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f26a96b5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26a41021d0 0x7f26a419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f26a96b5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26a419e1f0 con 0x7f26a4101810 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f26a96b5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f26a419e360 con 0x7f26a41021d0 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f26a96b5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26a419e520 con 0x7f26a4100620 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f269a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a4101810 0x7f26a4196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f269a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a4101810 0x7f26a4196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33856/0 (socket says 172.21.15.7:33856) 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a4100620 0x7f26a4196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f269a7fc640 1 -- 172.21.15.7:0/1134418386 learned_addr learned my addr 172.21.15.7:0/1134418386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f26a37fe640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26a41021d0 0x7f26a419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f269a7fc640 1 -- 172.21.15.7:0/1134418386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a4100620 msgr2=0x7f26a4196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f269a7fc640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a4100620 0x7f26a4196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f269a7fc640 1 -- 172.21.15.7:0/1134418386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26a41021d0 msgr2=0x7f26a419dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f269a7fc640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26a41021d0 0x7f26a419dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f269a7fc640 1 -- 172.21.15.7:0/1134418386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2694009560 con 0x7f26a4101810 2024-03-23T13:19:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f26a2ffd640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a4100620 0x7f26a4196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.737+0000 7f269a7fc640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a4101810 0x7f26a4196c40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f268c00e9f0 tx=0x7f268c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.738+0000 7f26a0ff9640 1 -- 172.21.15.7:0/1134418386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c009720 con 0x7f26a4101810 2024-03-23T13:19:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.738+0000 7f26a0ff9640 1 -- 172.21.15.7:0/1134418386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f268c00cd80 con 0x7f26a4101810 2024-03-23T13:19:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.738+0000 7f26a0ff9640 1 -- 172.21.15.7:0/1134418386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c010690 con 0x7f26a4101810 2024-03-23T13:19:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.738+0000 7f26a96b5640 1 -- 172.21.15.7:0/1134418386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a419e800 con 0x7f26a4101810 2024-03-23T13:19:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.738+0000 7f26a96b5640 1 -- 172.21.15.7:0/1134418386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a419ecc0 con 0x7f26a4101810 2024-03-23T13:19:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.740+0000 7f26a0ff9640 1 -- 172.21.15.7:0/1134418386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f268c0046c0 con 0x7f26a4101810 2024-03-23T13:19:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.740+0000 7f26ac372740 1 Processor -- start 2024-03-23T13:19:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26ac372740 1 -- start start 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26ac372740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec1125980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26ac372740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 0x55aec112aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26ac372740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec112b0c0 0x55aec112d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26ac372740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aec1094180 con 0x55aec1125560 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26ac372740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aec10916b0 con 0x55aec1125f50 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26ac372740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aec108ce90 con 0x55aec112b0c0 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec1125980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 0x55aec112aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f269a7fc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec112b0c0 0x55aec112d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec1125980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33870/0 (socket says 172.21.15.7:33870) 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a37fe640 1 -- 172.21.15.7:0/2173498985 learned_addr learned my addr 172.21.15.7:0/2173498985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a0ff9640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26900a2150 0x7f26900a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a37fe640 1 -- 172.21.15.7:0/2173498985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec112b0c0 msgr2=0x55aec112d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a37fe640 1 --2- 172.21.15.7:0/2173498985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec112b0c0 0x55aec112d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a37fe640 1 -- 172.21.15.7:0/2173498985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 msgr2=0x55aec112aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a37fe640 1 --2- 172.21.15.7:0/2173498985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 0x55aec112aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a37fe640 1 -- 172.21.15.7:0/2173498985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec1049d80 con 0x55aec1125560 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a2ffd640 1 --2- 172.21.15.7:0/2173498985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 0x55aec112aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a0ff9640 1 -- 172.21.15.7:0/1134418386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f268c014070 con 0x7f26a4101810 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.741+0000 7f26a37fe640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26900a2150 0x7f26900a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.742+0000 7f26a37fe640 1 --2- 172.21.15.7:0/2173498985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec1125980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f269c011590 tx=0x7f269c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601438a01ed3e90c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.742+0000 7f26997fa640 1 -- 172.21.15.7:0/2173498985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c00f840 con 0x55aec1125560 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.742+0000 7f26997fa640 1 -- 172.21.15.7:0/2173498985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f269c00fe80 con 0x55aec1125560 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.742+0000 7f26997fa640 1 -- 172.21.15.7:0/2173498985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c00eb60 con 0x55aec1125560 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.742+0000 7f26ac372740 1 -- 172.21.15.7:0/2173498985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 msgr2=0x55aec1125980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.742+0000 7f26a37fe640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26900a2150 0x7f26900a4610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f269c022000 tx=0x7f269c023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.742+0000 7f26ac372740 1 --2- 172.21.15.7:0/2173498985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec1125980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f269c011590 tx=0x7f269c011a50 comp rx=0 tx=0).stop 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 -- 172.21.15.7:0/2173498985 shutdown_connections 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 --2- 172.21.15.7:0/2173498985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec112b0c0 0x55aec112d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 --2- 172.21.15.7:0/2173498985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 0x55aec112aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 --2- 172.21.15.7:0/2173498985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec1125980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 -- 172.21.15.7:0/2173498985 >> 172.21.15.7:0/2173498985 conn(0x55aec1120ad0 msgr2=0x55aec1120ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 -- 172.21.15.7:0/2173498985 shutdown_connections 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 -- 172.21.15.7:0/2173498985 wait complete. 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 -- start start 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 Processor -- start 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec112f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125f50 0x55aec112fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec112b0c0 0x55aec1136a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aec1031d50 con 0x55aec1125560 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aec0d60310 con 0x55aec112b0c0 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aec10128f0 con 0x55aec1125f50 2024-03-23T13:19:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26ac372740 10 client.0 _ll_register_callbacks cb 0x55aec0d9dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec112f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec112f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33872/0 (socket says 172.21.15.7:33872) 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26a2ffd640 1 -- 172.21.15.7:0/3258640777 learned_addr learned my addr 172.21.15.7:0/3258640777 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f269a7fc640 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125f50 0x55aec112fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.743+0000 7f26a37fe640 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec112b0c0 0x55aec1136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f26a2ffd640 1 -- 172.21.15.7:0/3258640777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125f50 msgr2=0x55aec112fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f26a2ffd640 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125f50 0x55aec112fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f26a2ffd640 1 -- 172.21.15.7:0/3258640777 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec112b0c0 msgr2=0x55aec1136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f26a2ffd640 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec112b0c0 0x55aec1136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f26a2ffd640 1 -- 172.21.15.7:0/3258640777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c009cd0 con 0x55aec1125560 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f26a2ffd640 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec112f500 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2694009900 tx=0x7f2694004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc6b28ebcd1ee4e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f268a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f268a7fc640 1 -- 172.21.15.7:0/3258640777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694004df0 con 0x55aec1125560 2024-03-23T13:19:09.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f268a7fc640 1 -- 172.21.15.7:0/3258640777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2694004f50 con 0x55aec1125560 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f26ac372740 1 -- 172.21.15.7:0/3258640777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aec0ddd270 con 0x55aec1125560 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f268a7fc640 1 -- 172.21.15.7:0/3258640777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2694050030 con 0x55aec1125560 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f26ac372740 10 client.15105 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f26ac372740 1 -- 172.21.15.7:0/3258640777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aec108b510 con 0x55aec1125560 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f268a7fc640 1 -- 172.21.15.7:0/3258640777 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55aec0ddd270 con 0x55aec1125560 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f268a7fc640 1 -- 172.21.15.7:0/3258640777 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f269403c040 con 0x55aec1125560 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.744+0000 7f26ac372740 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.745+0000 7f26ac372740 10 client.15105 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.745+0000 7f26ac372740 20 client.15105 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.745+0000 7f26ac372740 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55aec113bbd0 0x55aec113d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.745+0000 7f26ac372740 4 client.15105 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.745+0000 7f26ac372740 1 -- 172.21.15.7:0/3258640777 --> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aec0dc9610 con 0x55aec113bbd0 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.745+0000 7f269a7fc640 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55aec113bbd0 0x55aec113d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.745+0000 7f268a7fc640 10 client.15105 ms_handle_connect on v2:172.21.15.155:6836/3999767587 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.745+0000 7f269a7fc640 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55aec113bbd0 0x55aec113d040 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=359a6a2102f24ad2 server_cookie=f01e20fcd39ac892 in_seq=0 out_seq=0 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f268a7fc640 1 -- 172.21.15.7:0/3258640777 <== mds.2 v2:172.21.15.155:6836/3999767587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55aec0dc9610 con 0x55aec113bbd0 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f268a7fc640 10 client.15105 handle_command_reply: tid=0 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 client.15105 shutdown 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 -- 172.21.15.7:0/3258640777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 msgr2=0x55aec112f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec112f500 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2694009900 tx=0x7f2694004270 comp rx=0 tx=0).stop 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 -- 172.21.15.7:0/3258640777 shutdown_connections 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55aec113bbd0 0x55aec113d040 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec112b0c0 0x55aec1136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125f50 0x55aec112fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f268a7fc640 0 client.15105 ms_handle_reset on v2:172.21.15.155:6836/3999767587 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 --2- 172.21.15.7:0/3258640777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125560 0x55aec112f500 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 -- 172.21.15.7:0/3258640777 >> 172.21.15.7:0/3258640777 conn(0x55aec1120ad0 msgr2=0x55aec111fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 -- 172.21.15.7:0/3258640777 shutdown_connections 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 1 -- 172.21.15.7:0/3258640777 wait complete. 2024-03-23T13:19:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.746+0000 7f26ac372740 20 client.15105 trim_cache size 0 max 16384 2024-03-23T13:19:09.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.764+0000 7f26ac372740 1 Processor -- start 2024-03-23T13:19:09.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.764+0000 7f26ac372740 1 -- start start 2024-03-23T13:19:09.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.764+0000 7f26ac372740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125560 0x55aec11245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.764+0000 7f26ac372740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125f50 0x55aec112a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.764+0000 7f26ac372740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec112b0c0 0x55aec111b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.764+0000 7f26ac372740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aec0dc9610 con 0x55aec1125f50 2024-03-23T13:19:09.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.764+0000 7f26ac372740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aec0ddd270 con 0x55aec112b0c0 2024-03-23T13:19:09.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.764+0000 7f26ac372740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aec1045f70 con 0x55aec1125560 2024-03-23T13:19:09.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f269a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125f50 0x55aec112a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f269a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125f50 0x55aec112a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33884/0 (socket says 172.21.15.7:33884) 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f269a7fc640 1 -- 172.21.15.7:0/4200783944 learned_addr learned my addr 172.21.15.7:0/4200783944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f26a2ffd640 1 --2- 172.21.15.7:0/4200783944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125560 0x55aec11245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f26a37fe640 1 --2- 172.21.15.7:0/4200783944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec112b0c0 0x55aec111b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f269a7fc640 1 -- 172.21.15.7:0/4200783944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125560 msgr2=0x55aec11245c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f269a7fc640 1 --2- 172.21.15.7:0/4200783944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125560 0x55aec11245c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f269a7fc640 1 -- 172.21.15.7:0/4200783944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec112b0c0 msgr2=0x55aec111b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f269a7fc640 1 --2- 172.21.15.7:0/4200783944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec112b0c0 0x55aec111b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f269a7fc640 1 -- 172.21.15.7:0/4200783944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec1027a80 con 0x55aec1125f50 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f269a7fc640 1 --2- 172.21.15.7:0/4200783944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125f50 0x55aec112a830 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f268c0cbb80 tx=0x7f268c0d0440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8c84a0067de081 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f2698ff9640 1 -- 172.21.15.7:0/4200783944 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c09a6d0 con 0x55aec1125f50 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f2698ff9640 1 -- 172.21.15.7:0/4200783944 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f268c09b470 con 0x55aec1125f50 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.765+0000 7f2698ff9640 1 -- 172.21.15.7:0/4200783944 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c099100 con 0x55aec1125f50 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 -- 172.21.15.7:0/4200783944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125f50 msgr2=0x55aec112a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 --2- 172.21.15.7:0/4200783944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125f50 0x55aec112a830 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f268c0cbb80 tx=0x7f268c0d0440 comp rx=0 tx=0).stop 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 -- 172.21.15.7:0/4200783944 shutdown_connections 2024-03-23T13:19:09.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 --2- 172.21.15.7:0/4200783944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec112b0c0 0x55aec111b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 --2- 172.21.15.7:0/4200783944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec1125f50 0x55aec112a830 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 --2- 172.21.15.7:0/4200783944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125560 0x55aec11245c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 -- 172.21.15.7:0/4200783944 >> 172.21.15.7:0/4200783944 conn(0x55aec1120ad0 msgr2=0x55aec11267e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 -- 172.21.15.7:0/4200783944 shutdown_connections 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 -- 172.21.15.7:0/4200783944 wait complete. 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 -- start start 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 Processor -- start 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125560 0x55aec1134de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 0x55aec1135320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec112b0c0 0x55aec1135be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aec1136120 con 0x55aec112b0c0 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aec1136290 con 0x55aec1125f50 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.766+0000 7f26ac372740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aec1136450 con 0x55aec1125560 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26ac372740 10 client.0 _ll_register_callbacks cb 0x7f2694009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec112b0c0 0x55aec1135be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec112b0c0 0x55aec1135be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33896/0 (socket says 172.21.15.7:33896) 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125560 0x55aec1134de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f269a7fc640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 0x55aec1135320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26a37fe640 1 -- 172.21.15.7:0/3029194378 learned_addr learned my addr 172.21.15.7:0/3029194378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26a37fe640 1 -- 172.21.15.7:0/3029194378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125560 msgr2=0x55aec1134de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26a37fe640 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125560 0x55aec1134de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26a37fe640 1 -- 172.21.15.7:0/3029194378 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 msgr2=0x55aec1135320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26a37fe640 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 0x55aec1135320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26a37fe640 1 -- 172.21.15.7:0/3029194378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c0cd040 con 0x55aec112b0c0 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26a37fe640 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec112b0c0 0x55aec1135be0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55aec11267b0 tx=0x7f269c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99a92019f68bab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f268a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f268a7fc640 1 -- 172.21.15.7:0/3029194378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c00eee0 con 0x55aec112b0c0 2024-03-23T13:19:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f268a7fc640 1 -- 172.21.15.7:0/3029194378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f269c009500 con 0x55aec112b0c0 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26ac372740 1 -- 172.21.15.7:0/3029194378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aec0d8c6c0 con 0x55aec112b0c0 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f268a7fc640 1 -- 172.21.15.7:0/3029194378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f269c003eb0 con 0x55aec112b0c0 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26ac372740 10 client.15111 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f26ac372740 1 -- 172.21.15.7:0/3029194378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aec104d7e0 con 0x55aec112b0c0 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.767+0000 7f268a7fc640 1 -- 172.21.15.7:0/3029194378 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f269c044020 con 0x55aec112b0c0 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f268a7fc640 1 -- 172.21.15.7:0/3029194378 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f269c031070 con 0x55aec112b0c0 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f26ac372740 10 client.15111 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f26ac372740 10 client.15111 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f26ac372740 20 client.15111 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f26ac372740 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55aec11a8480 0x55aec11a7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f26ac372740 4 client.15111 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f26ac372740 1 -- 172.21.15.7:0/3029194378 --> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] -- command(tid 0: {"prefix": "status"}) -- 0x55aec0d385e0 con 0x55aec11a8480 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f26a2ffd640 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55aec11a8480 0x55aec11a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f268a7fc640 10 client.15111 ms_handle_connect on v2:172.21.15.155:6836/3999767587 2024-03-23T13:19:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f26a2ffd640 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55aec11a8480 0x55aec11a7190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41d076e8b31f750 server_cookie=b44d52112f5b73b2 in_seq=0 out_seq=0 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f268a7fc640 1 -- 172.21.15.7:0/3029194378 <== mds.2 v2:172.21.15.155:6836/3999767587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55aec0d385e0 con 0x55aec11a8480 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f268a7fc640 10 client.15111 handle_command_reply: tid=0 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.768+0000 7f26ac372740 1 client.15111 shutdown 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 1 -- 172.21.15.7:0/3029194378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec112b0c0 msgr2=0x55aec1135be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec112b0c0 0x55aec1135be0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55aec11267b0 tx=0x7f269c00eb80 comp rx=0 tx=0).stop 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 1 -- 172.21.15.7:0/3029194378 shutdown_connections 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55aec11a8480 0x55aec11a7190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aec112b0c0 0x55aec1135be0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aec1125f50 0x55aec1135320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f268a7fc640 0 client.15111 ms_handle_reset on v2:172.21.15.155:6836/3999767587 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 1 --2- 172.21.15.7:0/3029194378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aec1125560 0x55aec1134de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 1 -- 172.21.15.7:0/3029194378 >> 172.21.15.7:0/3029194378 conn(0x55aec1120ad0 msgr2=0x55aec11245c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 1 -- 172.21.15.7:0/3029194378 shutdown_connections 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 1 -- 172.21.15.7:0/3029194378 wait complete. 2024-03-23T13:19:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26ac372740 20 client.15111 trim_cache size 0 max 16384 2024-03-23T13:19:09.776 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:19:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:19:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-03-23T13:19:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24475, 2024-03-23T13:19:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-23T13:19:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-23T13:19:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-23T13:19:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 84.216722828000002, 2024-03-23T13:19:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-03-23T13:19:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-23T13:19:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T13:19:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 107.00429924399999 2024-03-23T13:19:09.777 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:19:09.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 -- 172.21.15.7:0/1134418386 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26900a2150 msgr2=0x7f26900a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26900a2150 0x7f26900a4610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f269c022000 tx=0x7f269c023000 comp rx=0 tx=0).stop 2024-03-23T13:19:09.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 -- 172.21.15.7:0/1134418386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a4101810 msgr2=0x7f26a4196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a4101810 0x7f26a4196c40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f268c00e9f0 tx=0x7f268c00edf0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 -- 172.21.15.7:0/1134418386 shutdown_connections 2024-03-23T13:19:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26900a2150 0x7f26900a4610 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26a41021d0 0x7f26a419dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a4101810 0x7f26a4196c40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 --2- 172.21.15.7:0/1134418386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a4100620 0x7f26a4196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 -- 172.21.15.7:0/1134418386 >> 172.21.15.7:0/1134418386 conn(0x7f26a40fbbb0 msgr2=0x7f26a4104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 -- 172.21.15.7:0/1134418386 shutdown_connections 2024-03-23T13:19:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.769+0000 7f26a96b5640 1 -- 172.21.15.7:0/1134418386 wait complete. 2024-03-23T13:19:09.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-23T13:19:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.883+0000 7f424446f640 1 -- 172.21.15.7:0/3158732049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f423c100630 msgr2=0x7f423c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.883+0000 7f424446f640 1 --2- 172.21.15.7:0/3158732049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f423c100630 0x7f423c100a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4238009900 tx=0x7f423802bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.883+0000 7f424446f640 1 -- 172.21.15.7:0/3158732049 shutdown_connections 2024-03-23T13:19:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.883+0000 7f424446f640 1 --2- 172.21.15.7:0/3158732049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423c1021e0 0x7f423c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.883+0000 7f424446f640 1 --2- 172.21.15.7:0/3158732049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f423c101820 0x7f423c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.883+0000 7f424446f640 1 --2- 172.21.15.7:0/3158732049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f423c100630 0x7f423c100a50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.883+0000 7f424446f640 1 -- 172.21.15.7:0/3158732049 >> 172.21.15.7:0/3158732049 conn(0x7f423c0fbbc0 msgr2=0x7f423c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.883+0000 7f424446f640 1 -- 172.21.15.7:0/3158732049 shutdown_connections 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.883+0000 7f424446f640 1 -- 172.21.15.7:0/3158732049 wait complete. 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424446f640 1 Processor -- start 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424446f640 1 -- start start 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424446f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f423c100630 0x7f423c06a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424446f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f423c101820 0x7f423c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424446f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423c1021e0 0x7f423c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424446f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f423c19f320 con 0x7f423c100630 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424446f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f423c19f490 con 0x7f423c101820 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424446f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f423c19f650 con 0x7f423c1021e0 2024-03-23T13:19:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424220b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f423c100630 0x7f423c06a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424220b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f423c100630 0x7f423c06a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33914/0 (socket says 172.21.15.7:33914) 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424220b640 1 -- 172.21.15.7:0/694034994 learned_addr learned my addr 172.21.15.7:0/694034994 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f4241a0a640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f423c101820 0x7f423c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f4242a0c640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423c1021e0 0x7f423c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424220b640 1 -- 172.21.15.7:0/694034994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423c1021e0 msgr2=0x7f423c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424220b640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423c1021e0 0x7f423c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424220b640 1 -- 172.21.15.7:0/694034994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f423c101820 msgr2=0x7f423c19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424220b640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f423c101820 0x7f423c19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.884+0000 7f424220b640 1 -- 172.21.15.7:0/694034994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4238009560 con 0x7f423c100630 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.885+0000 7f424220b640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f423c100630 0x7f423c06a850 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4238005ab0 tx=0x7f4238005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.885+0000 7f42337fe640 1 -- 172.21.15.7:0/694034994 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380043d0 con 0x7f423c100630 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.885+0000 7f42337fe640 1 -- 172.21.15.7:0/694034994 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42380052d0 con 0x7f423c100630 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.885+0000 7f424446f640 1 -- 172.21.15.7:0/694034994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c19f870 con 0x7f423c100630 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.885+0000 7f42337fe640 1 -- 172.21.15.7:0/694034994 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238003d40 con 0x7f423c100630 2024-03-23T13:19:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.885+0000 7f424446f640 1 -- 172.21.15.7:0/694034994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c19fdd0 con 0x7f423c100630 2024-03-23T13:19:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.887+0000 7f42337fe640 1 -- 172.21.15.7:0/694034994 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f4238005440 con 0x7f423c100630 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.887+0000 7f4247054740 1 Processor -- start 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4247054740 1 -- start start 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4247054740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21560 0x55d5f3d21980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4247054740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 0x55d5f3d26a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4247054740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d270c0 0x55d5f3d29550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4247054740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d5f3c90180 con 0x55d5f3d21f50 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4247054740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d5f3c8d6b0 con 0x55d5f3d21560 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4247054740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5f3c88e90 con 0x55d5f3d270c0 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f42337fe640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f42280a21d0 0x7f42280a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4242a0c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21560 0x55d5f3d21980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4241a0a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d270c0 0x55d5f3d29550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f424220b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 0x55d5f3d26a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4241a0a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d270c0 0x55d5f3d29550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:36670/0 (socket says 172.21.15.7:36670) 2024-03-23T13:19:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4242a0c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21560 0x55d5f3d21980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:51132/0 (socket says 172.21.15.7:51132) 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4241a0a640 1 -- 172.21.15.7:0/1368897940 learned_addr learned my addr 172.21.15.7:0/1368897940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4242a0c640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f42280a21d0 0x7f42280a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f42337fe640 1 -- 172.21.15.7:0/694034994 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f42380efdb0 con 0x7f423c100630 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4241a0a640 1 -- 172.21.15.7:0/1368897940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21560 msgr2=0x55d5f3d21980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4241a0a640 1 --2- 172.21.15.7:0/1368897940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21560 0x55d5f3d21980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4241a0a640 1 -- 172.21.15.7:0/1368897940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 msgr2=0x55d5f3d26a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4241a0a640 1 --2- 172.21.15.7:0/1368897940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 0x55d5f3d26a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.888+0000 7f4241a0a640 1 -- 172.21.15.7:0/1368897940 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f3c45d80 con 0x55d5f3d270c0 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.889+0000 7f4242a0c640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f42280a21d0 0x7f42280a4690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55d5f3d227b0 tx=0x7f4234009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.889+0000 7f4241a0a640 1 --2- 172.21.15.7:0/1368897940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d270c0 0x55d5f3d29550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f422c009c90 tx=0x7f422c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95971f4592b1ff93 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.889+0000 7f4230bfe640 1 -- 172.21.15.7:0/1368897940 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c019630 con 0x55d5f3d270c0 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.889+0000 7f4230bfe640 1 -- 172.21.15.7:0/1368897940 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f422c009e40 con 0x55d5f3d270c0 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.889+0000 7f4230bfe640 1 -- 172.21.15.7:0/1368897940 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c00eb30 con 0x55d5f3d270c0 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.889+0000 7f4247054740 1 -- 172.21.15.7:0/1368897940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d270c0 msgr2=0x55d5f3d29550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.889+0000 7f4247054740 1 --2- 172.21.15.7:0/1368897940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d270c0 0x55d5f3d29550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f422c009c90 tx=0x7f422c011810 comp rx=0 tx=0).stop 2024-03-23T13:19:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 -- 172.21.15.7:0/1368897940 shutdown_connections 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 --2- 172.21.15.7:0/1368897940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d270c0 0x55d5f3d29550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 --2- 172.21.15.7:0/1368897940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 0x55d5f3d26a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 --2- 172.21.15.7:0/1368897940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21560 0x55d5f3d21980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 -- 172.21.15.7:0/1368897940 >> 172.21.15.7:0/1368897940 conn(0x55d5f3d1cad0 msgr2=0x55d5f3d1ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 -- 172.21.15.7:0/1368897940 shutdown_connections 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 -- 172.21.15.7:0/1368897940 wait complete. 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 -- start start 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 Processor -- start 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21560 0x55d5f3d2b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 0x55d5f3d2ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d270c0 0x55d5f3d32ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d5f3c2dd50 con 0x55d5f3d21f50 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d5f395c310 con 0x55d5f3d270c0 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5f3c0e8f0 con 0x55d5f3d21560 2024-03-23T13:19:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4247054740 10 client.0 _ll_register_callbacks cb 0x55d5f3999ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4241a0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 0x55d5f3d2ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4241a0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 0x55d5f3d2ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33938/0 (socket says 172.21.15.7:33938) 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4241a0a640 1 -- 172.21.15.7:0/3590028808 learned_addr learned my addr 172.21.15.7:0/3590028808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f424220b640 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21560 0x55d5f3d2b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.890+0000 7f4242a0c640 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d270c0 0x55d5f3d32ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4241a0a640 1 -- 172.21.15.7:0/3590028808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21560 msgr2=0x55d5f3d2b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4241a0a640 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21560 0x55d5f3d2b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4241a0a640 1 -- 172.21.15.7:0/3590028808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d270c0 msgr2=0x55d5f3d32ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4241a0a640 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d270c0 0x55d5f3d32ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4241a0a640 1 -- 172.21.15.7:0/3590028808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422c0098f0 con 0x55d5f3d21f50 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4241a0a640 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 0x55d5f3d2ba70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f422c011d10 tx=0x7f422c00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72bac9dabf83d49 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4225ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4225ffb640 1 -- 172.21.15.7:0/3590028808 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c005450 con 0x55d5f3d21f50 2024-03-23T13:19:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4225ffb640 1 -- 172.21.15.7:0/3590028808 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f422c0055b0 con 0x55d5f3d21f50 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4247054740 1 -- 172.21.15.7:0/3590028808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d5f39886c0 con 0x55d5f3d21f50 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4225ffb640 1 -- 172.21.15.7:0/3590028808 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f422c0046c0 con 0x55d5f3d21f50 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4247054740 10 client.15123 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4247054740 1 -- 172.21.15.7:0/3590028808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5f3c87510 con 0x55d5f3d21f50 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4225ffb640 1 -- 172.21.15.7:0/3590028808 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d5f39886c0 con 0x55d5f3d21f50 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4225ffb640 1 -- 172.21.15.7:0/3590028808 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f422c025a50 con 0x55d5f3d21f50 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.891+0000 7f4247054740 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.892+0000 7f4247054740 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.892+0000 7f4247054740 20 client.15123 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.892+0000 7f4247054740 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55d5f3d37c90 0x55d5f3d39100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.892+0000 7f4247054740 4 client.15123 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.892+0000 7f4247054740 1 -- 172.21.15.7:0/3590028808 --> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5f39d9270 con 0x55d5f3d37c90 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.892+0000 7f424220b640 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55d5f3d37c90 0x55d5f3d39100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.892+0000 7f424220b640 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55d5f3d37c90 0x55d5f3d39100 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4eff606f8466ffe server_cookie=dcb22a88f747d978 in_seq=0 out_seq=0 2024-03-23T13:19:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.892+0000 7f4225ffb640 10 client.15123 ms_handle_connect on v2:172.21.15.155:6838/131946068 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4225ffb640 1 -- 172.21.15.7:0/3590028808 <== mds.1 v2:172.21.15.155:6838/131946068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d5f39d9270 con 0x55d5f3d37c90 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4225ffb640 10 client.15123 handle_command_reply: tid=0 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 client.15123 shutdown 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 -- 172.21.15.7:0/3590028808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 msgr2=0x55d5f3d2ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 0x55d5f3d2ba70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f422c011d10 tx=0x7f422c00ef50 comp rx=0 tx=0).stop 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 -- 172.21.15.7:0/3590028808 shutdown_connections 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55d5f3d37c90 0x55d5f3d39100 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d270c0 0x55d5f3d32ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21f50 0x55d5f3d2ba70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4225ffb640 0 client.15123 ms_handle_reset on v2:172.21.15.155:6838/131946068 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 --2- 172.21.15.7:0/3590028808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21560 0x55d5f3d2b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 -- 172.21.15.7:0/3590028808 >> 172.21.15.7:0/3590028808 conn(0x55d5f3d1cad0 msgr2=0x55d5f3d1bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 -- 172.21.15.7:0/3590028808 shutdown_connections 2024-03-23T13:19:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 1 -- 172.21.15.7:0/3590028808 wait complete. 2024-03-23T13:19:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.893+0000 7f4247054740 20 client.15123 trim_cache size 0 max 16384 2024-03-23T13:19:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4247054740 1 Processor -- start 2024-03-23T13:19:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4247054740 1 -- start start 2024-03-23T13:19:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4247054740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d1cad0 0x55d5f3d17860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4247054740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21560 0x55d5f3d32a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4247054740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21f50 0x55d5f3d1ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4247054740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d5f39886f0 con 0x55d5f3d1cad0 2024-03-23T13:19:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4247054740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d5f3d22670 con 0x55d5f3d21f50 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4247054740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5f3c41f70 con 0x55d5f3d21560 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f424220b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d1cad0 0x55d5f3d17860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f424220b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d1cad0 0x55d5f3d17860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33946/0 (socket says 172.21.15.7:33946) 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f424220b640 1 -- 172.21.15.7:0/3917463834 learned_addr learned my addr 172.21.15.7:0/3917463834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4242a0c640 1 --2- 172.21.15.7:0/3917463834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21f50 0x55d5f3d1ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4241a0a640 1 --2- 172.21.15.7:0/3917463834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21560 0x55d5f3d32a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f424220b640 1 -- 172.21.15.7:0/3917463834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21560 msgr2=0x55d5f3d32a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f424220b640 1 --2- 172.21.15.7:0/3917463834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21560 0x55d5f3d32a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f424220b640 1 -- 172.21.15.7:0/3917463834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21f50 msgr2=0x55d5f3d1ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f424220b640 1 --2- 172.21.15.7:0/3917463834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21f50 0x55d5f3d1ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f424220b640 1 -- 172.21.15.7:0/3917463834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f3c23a80 con 0x55d5f3d1cad0 2024-03-23T13:19:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.911+0000 7f4242a0c640 1 --2- 172.21.15.7:0/3917463834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21f50 0x55d5f3d1ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f424220b640 1 --2- 172.21.15.7:0/3917463834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d1cad0 0x55d5f3d17860 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f423803afd0 tx=0x7f423808fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd55f15e355ef6d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4227fff640 1 -- 172.21.15.7:0/3917463834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380bc870 con 0x55d5f3d1cad0 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4227fff640 1 -- 172.21.15.7:0/3917463834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42380bc5e0 con 0x55d5f3d1cad0 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4227fff640 1 -- 172.21.15.7:0/3917463834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380ed9c0 con 0x55d5f3d1cad0 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4247054740 1 -- 172.21.15.7:0/3917463834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d1cad0 msgr2=0x55d5f3d17860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4247054740 1 --2- 172.21.15.7:0/3917463834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d1cad0 0x55d5f3d17860 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f423803afd0 tx=0x7f423808fb60 comp rx=0 tx=0).stop 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4247054740 1 -- 172.21.15.7:0/3917463834 shutdown_connections 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4247054740 1 --2- 172.21.15.7:0/3917463834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d21f50 0x55d5f3d1ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4247054740 1 --2- 172.21.15.7:0/3917463834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21560 0x55d5f3d32a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4247054740 1 --2- 172.21.15.7:0/3917463834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d1cad0 0x55d5f3d17860 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4247054740 1 -- 172.21.15.7:0/3917463834 >> 172.21.15.7:0/3917463834 conn(0x55d5f3d26bc0 msgr2=0x55d5f3d36cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4247054740 1 -- 172.21.15.7:0/3917463834 shutdown_connections 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.912+0000 7f4247054740 1 -- 172.21.15.7:0/3917463834 wait complete. 2024-03-23T13:19:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4247054740 1 -- start start 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4247054740 1 Processor -- start 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4247054740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d1cad0 0x55d5f3d2df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4247054740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21560 0x55d5f3d2e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4247054740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21f50 0x55d5f3d31430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4247054740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d5f3d31970 con 0x55d5f3d21560 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4247054740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d5f3d31ae0 con 0x55d5f3d1cad0 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4247054740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5f3d31ca0 con 0x55d5f3d21f50 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4247054740 10 client.0 _ll_register_callbacks cb 0x7f422c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4241a0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21560 0x55d5f3d2e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4241a0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21560 0x55d5f3d2e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33954/0 (socket says 172.21.15.7:33954) 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4241a0a640 1 -- 172.21.15.7:0/3027299750 learned_addr learned my addr 172.21.15.7:0/3027299750 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4242a0c640 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21f50 0x55d5f3d31430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f424220b640 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d1cad0 0x55d5f3d2df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4241a0a640 1 -- 172.21.15.7:0/3027299750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21f50 msgr2=0x55d5f3d31430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4241a0a640 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21f50 0x55d5f3d31430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4241a0a640 1 -- 172.21.15.7:0/3027299750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d1cad0 msgr2=0x55d5f3d2df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4241a0a640 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d1cad0 0x55d5f3d2df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4241a0a640 1 -- 172.21.15.7:0/3027299750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c19fdd0 con 0x55d5f3d21560 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4241a0a640 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21560 0x55d5f3d2e460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f422c025d80 tx=0x7f422c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd919c408f0fe18e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.913+0000 7f4225ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4225ffb640 1 -- 172.21.15.7:0/3027299750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c005660 con 0x55d5f3d21560 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4225ffb640 1 -- 172.21.15.7:0/3027299750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f422c000e40 con 0x55d5f3d21560 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4247054740 1 -- 172.21.15.7:0/3027299750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f422c009e40 con 0x55d5f3d21560 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4225ffb640 1 -- 172.21.15.7:0/3027299750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f422c03f030 con 0x55d5f3d21560 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4247054740 10 client.15129 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4247054740 1 -- 172.21.15.7:0/3027299750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5f3c497e0 con 0x55d5f3d21560 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4225ffb640 1 -- 172.21.15.7:0/3027299750 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f422c009e40 con 0x55d5f3d21560 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4225ffb640 1 -- 172.21.15.7:0/3027299750 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f422c025830 con 0x55d5f3d21560 2024-03-23T13:19:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4247054740 10 client.15129 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4247054740 10 client.15129 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4247054740 20 client.15129 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4247054740 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55d5f3d270c0 0x55d5f3da0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4247054740 4 client.15129 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f4247054740 1 -- 172.21.15.7:0/3027299750 --> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] -- command(tid 0: {"prefix": "status"}) -- 0x55d5f39c5610 con 0x55d5f3d270c0 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.914+0000 7f424220b640 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55d5f3d270c0 0x55d5f3da0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f424220b640 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55d5f3d270c0 0x55d5f3da0850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa841bf2647fe7b2 server_cookie=d9e1cb018629e2ea in_seq=0 out_seq=0 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4225ffb640 10 client.15129 ms_handle_connect on v2:172.21.15.155:6838/131946068 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4225ffb640 1 -- 172.21.15.7:0/3027299750 <== mds.1 v2:172.21.15.155:6838/131946068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d5f39c5610 con 0x55d5f3d270c0 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4225ffb640 10 client.15129 handle_command_reply: tid=0 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 client.15129 shutdown 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 -- 172.21.15.7:0/3027299750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21560 msgr2=0x55d5f3d2e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21560 0x55d5f3d2e460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f422c025d80 tx=0x7f422c005d30 comp rx=0 tx=0).stop 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 -- 172.21.15.7:0/3027299750 shutdown_connections 2024-03-23T13:19:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55d5f3d270c0 0x55d5f3da0850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5f3d21f50 0x55d5f3d31430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5f3d21560 0x55d5f3d2e460 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4225ffb640 0 client.15129 ms_handle_reset on v2:172.21.15.155:6838/131946068 2024-03-23T13:19:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 --2- 172.21.15.7:0/3027299750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5f3d1cad0 0x55d5f3d2df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 -- 172.21.15.7:0/3027299750 >> 172.21.15.7:0/3027299750 conn(0x55d5f3d26bc0 msgr2=0x55d5f3d205c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 -- 172.21.15.7:0/3027299750 shutdown_connections 2024-03-23T13:19:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 1 -- 172.21.15.7:0/3027299750 wait complete. 2024-03-23T13:19:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.915+0000 7f4247054740 20 client.15129 trim_cache size 0 max 16384 2024-03-23T13:19:09.923 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:19:09.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24496, 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 85.369097089999997, 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 101.58926800499999 2024-03-23T13:19:09.924 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:19:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 -- 172.21.15.7:0/694034994 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f42280a21d0 msgr2=0x7f42280a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f42280a21d0 0x7f42280a4690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55d5f3d227b0 tx=0x7f4234009500 comp rx=0 tx=0).stop 2024-03-23T13:19:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 -- 172.21.15.7:0/694034994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f423c100630 msgr2=0x7f423c06a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f423c100630 0x7f423c06a850 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4238005ab0 tx=0x7f4238005e50 comp rx=0 tx=0).stop 2024-03-23T13:19:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 -- 172.21.15.7:0/694034994 shutdown_connections 2024-03-23T13:19:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f42280a21d0 0x7f42280a4690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423c1021e0 0x7f423c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f423c101820 0x7f423c19c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 --2- 172.21.15.7:0/694034994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f423c100630 0x7f423c06a850 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:09.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 -- 172.21.15.7:0/694034994 >> 172.21.15.7:0/694034994 conn(0x7f423c0fbbc0 msgr2=0x7f423c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:09.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 -- 172.21.15.7:0/694034994 shutdown_connections 2024-03-23T13:19:09.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:09.916+0000 7f424446f640 1 -- 172.21.15.7:0/694034994 wait complete. 2024-03-23T13:19:09.926 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:09 smithi007 ceph-mon[32184]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:19:09.926 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:09 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4090573075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:09.926 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:09 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4048920169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:09.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T13:19:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.030+0000 7fb3c7993640 1 -- 172.21.15.7:0/1137011536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c01021c0 msgr2=0x7fb3c0108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.030+0000 7fb3c7993640 1 --2- 172.21.15.7:0/1137011536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c01021c0 0x7fb3c0108a50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009ca0 tx=0x7fb3b802c0f0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.030+0000 7fb3c7993640 1 -- 172.21.15.7:0/1137011536 shutdown_connections 2024-03-23T13:19:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.030+0000 7fb3c7993640 1 --2- 172.21.15.7:0/1137011536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c01021c0 0x7fb3c0108a50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.030+0000 7fb3c7993640 1 --2- 172.21.15.7:0/1137011536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3c0101800 0x7fb3c0101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.030+0000 7fb3c7993640 1 --2- 172.21.15.7:0/1137011536 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3c0100610 0x7fb3c0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.030+0000 7fb3c7993640 1 -- 172.21.15.7:0/1137011536 >> 172.21.15.7:0/1137011536 conn(0x7fb3c00fbba0 msgr2=0x7fb3c00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c7993640 1 -- 172.21.15.7:0/1137011536 shutdown_connections 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c7993640 1 -- 172.21.15.7:0/1137011536 wait complete. 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c7993640 1 Processor -- start 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c7993640 1 -- start start 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c7993640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0100610 0x7fb3c01977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c7993640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3c0101800 0x7fb3c0197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c7993640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3c01021c0 0x7fb3c019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c7993640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c019f290 con 0x7fb3c0100610 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c7993640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c019f400 con 0x7fb3c01021c0 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c7993640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c019f5c0 con 0x7fb3c0101800 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c572f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0100610 0x7fb3c01977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c572f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0100610 0x7fb3c01977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33976/0 (socket says 172.21.15.7:33976) 2024-03-23T13:19:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c572f640 1 -- 172.21.15.7:0/950683279 learned_addr learned my addr 172.21.15.7:0/950683279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3c0101800 0x7fb3c0197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.031+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3c01021c0 0x7fb3c019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3c572f640 1 -- 172.21.15.7:0/950683279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3c0101800 msgr2=0x7fb3c0197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3c572f640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3c0101800 0x7fb3c0197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3c572f640 1 -- 172.21.15.7:0/950683279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3c01021c0 msgr2=0x7fb3c019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3c572f640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3c01021c0 0x7fb3c019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3c572f640 1 -- 172.21.15.7:0/950683279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b8009900 con 0x7fb3c0100610 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3c572f640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0100610 0x7fb3c01977a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00e9c0 tx=0x7fb3bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3b67fc640 1 -- 172.21.15.7:0/950683279 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc009700 con 0x7fb3c0100610 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3b67fc640 1 -- 172.21.15.7:0/950683279 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3bc00cd80 con 0x7fb3c0100610 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3c7993640 1 -- 172.21.15.7:0/950683279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c019f8a0 con 0x7fb3c0100610 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3b67fc640 1 -- 172.21.15.7:0/950683279 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc010430 con 0x7fb3c0100610 2024-03-23T13:19:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.032+0000 7fb3c7993640 1 -- 172.21.15.7:0/950683279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c019fdb0 con 0x7fb3c0100610 2024-03-23T13:19:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.034+0000 7fb3b67fc640 1 -- 172.21.15.7:0/950683279 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7fb3bc003680 con 0x7fb3c0100610 2024-03-23T13:19:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3ca574740 1 Processor -- start 2024-03-23T13:19:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3ca574740 1 -- start start 2024-03-23T13:19:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0ea560 0x5580ff0ea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3b67fc640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb3ac0a2150 0x7fb3ac0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0efa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0f2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3ca574740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5580ff059180 con 0x5580ff0f00c0 2024-03-23T13:19:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3ca574740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5580ff0566b0 con 0x5580ff0eaf50 2024-03-23T13:19:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3ca574740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5580ff051e90 con 0x5580ff0ea560 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c572f640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb3ac0a2150 0x7fb3ac0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c5f30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c5f30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33978/0 (socket says 172.21.15.7:33978) 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c5f30640 1 -- 172.21.15.7:0/32101611 learned_addr learned my addr 172.21.15.7:0/32101611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/32101611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0efa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/32101611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0ea560 0x5580ff0ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3b67fc640 1 -- 172.21.15.7:0/950683279 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb3bc014070 con 0x7fb3c0100610 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c5f30640 1 -- 172.21.15.7:0/32101611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0ea560 msgr2=0x5580ff0ea980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/32101611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0ea560 0x5580ff0ea980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c5f30640 1 -- 172.21.15.7:0/32101611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 msgr2=0x5580ff0efa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/32101611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0efa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c5f30640 1 -- 172.21.15.7:0/32101611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580ff00ed80 con 0x5580ff0f00c0 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.035+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/32101611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0efa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3c572f640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb3ac0a2150 0x7fb3ac0a4610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc09aab0 tx=0x7fb3bc06cdc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/32101611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0f2550 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80325e0 tx=0x7fb3b802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a520d2d38761e235 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3ab7fe640 1 -- 172.21.15.7:0/32101611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80346d0 con 0x5580ff0f00c0 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3ab7fe640 1 -- 172.21.15.7:0/32101611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8034a90 con 0x5580ff0f00c0 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3ab7fe640 1 -- 172.21.15.7:0/32101611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8003d70 con 0x5580ff0f00c0 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3ca574740 1 -- 172.21.15.7:0/32101611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 msgr2=0x5580ff0f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3ca574740 1 --2- 172.21.15.7:0/32101611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0f2550 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80325e0 tx=0x7fb3b802c0f0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3ca574740 1 -- 172.21.15.7:0/32101611 shutdown_connections 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3ca574740 1 --2- 172.21.15.7:0/32101611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0f2550 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3ca574740 1 --2- 172.21.15.7:0/32101611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0efa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3ca574740 1 --2- 172.21.15.7:0/32101611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0ea560 0x5580ff0ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.036+0000 7fb3ca574740 1 -- 172.21.15.7:0/32101611 >> 172.21.15.7:0/32101611 conn(0x5580ff0e5ad0 msgr2=0x5580ff0e5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 1 -- 172.21.15.7:0/32101611 shutdown_connections 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 1 -- 172.21.15.7:0/32101611 wait complete. 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 1 -- start start 2024-03-23T13:19:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 1 Processor -- start 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0f4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0f00c0 0x5580ff0fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5580feff6d50 con 0x5580ff0ea560 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5580fed25310 con 0x5580ff0eaf50 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5580fefd78f0 con 0x5580ff0f00c0 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3ca574740 10 client.0 _ll_register_callbacks cb 0x5580fed62ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3c5f30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3c5f30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0f4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33988/0 (socket says 172.21.15.7:33988) 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3c5f30640 1 -- 172.21.15.7:0/2887523725 learned_addr learned my addr 172.21.15.7:0/2887523725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.037+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0f00c0 0x5580ff0fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3c5f30640 1 -- 172.21.15.7:0/2887523725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0f00c0 msgr2=0x5580ff0fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0f00c0 0x5580ff0fba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3c5f30640 1 -- 172.21.15.7:0/2887523725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 msgr2=0x5580ff0f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3c5f30640 1 -- 172.21.15.7:0/2887523725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c0109350 con 0x5580ff0ea560 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0f4500 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009600 tx=0x7fb3b80323e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c65414285aa4380 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/2887523725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8040900 con 0x5580ff0ea560 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/2887523725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb3b8040a60 con 0x5580ff0ea560 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3ca574740 1 -- 172.21.15.7:0/2887523725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580fed516c0 con 0x5580ff0ea560 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/2887523725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3b8040c20 con 0x5580ff0ea560 2024-03-23T13:19:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3ca574740 10 client.15141 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3ca574740 1 -- 172.21.15.7:0/2887523725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580ff050510 con 0x5580ff0ea560 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/2887523725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5580fed516c0 con 0x5580ff0ea560 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/2887523725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b804b050 con 0x5580ff0ea560 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3ca574740 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3ca574740 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3ca574740 20 client.15141 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3ca574740 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5580ff0ffbe0 0x5580ff101dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3ca574740 4 client.15141 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.038+0000 7fb3ca574740 1 -- 172.21.15.7:0/2887523725 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580feda2270 con 0x5580ff0ffbe0 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.039+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5580ff0ffbe0 0x5580ff101dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.039+0000 7fb3a8ff9640 10 client.15141 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.039+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5580ff0ffbe0 0x5580ff101dc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee9ec716b9c9d6e7 server_cookie=d054a3b4884015dc in_seq=0 out_seq=0 2024-03-23T13:19:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/2887523725 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5580feda2270 con 0x5580ff0ffbe0 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3a8ff9640 10 client.15141 handle_command_reply: tid=0 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 client.15141 shutdown 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 -- 172.21.15.7:0/2887523725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 msgr2=0x5580ff0f4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0f4500 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009600 tx=0x7fb3b80323e0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 -- 172.21.15.7:0/2887523725 shutdown_connections 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5580ff0ffbe0 0x5580ff101dc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0f00c0 0x5580ff0fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3a8ff9640 0 client.15141 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 --2- 172.21.15.7:0/2887523725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0f4500 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 -- 172.21.15.7:0/2887523725 >> 172.21.15.7:0/2887523725 conn(0x5580ff0e5ad0 msgr2=0x5580ff0e4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 -- 172.21.15.7:0/2887523725 shutdown_connections 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 1 -- 172.21.15.7:0/2887523725 wait complete. 2024-03-23T13:19:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.040+0000 7fb3ca574740 20 client.15141 trim_cache size 0 max 16384 2024-03-23T13:19:10.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:09 smithi155 ceph-mon[36157]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:19:10.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:09 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/4090573075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:10.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:09 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/4048920169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.058+0000 7fb3ca574740 1 Processor -- start 2024-03-23T13:19:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.058+0000 7fb3ca574740 1 -- start start 2024-03-23T13:19:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0eaf50 0x5580ff0fba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0f00c0 0x5580ff0f4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3ca574740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5580ff0eb640 con 0x5580ff0ea560 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3ca574740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5580feda2270 con 0x5580ff0f00c0 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3ca574740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5580fed516c0 con 0x5580ff0eaf50 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c5f30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c5f30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0e5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34004/0 (socket says 172.21.15.7:34004) 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c5f30640 1 -- 172.21.15.7:0/1137275011 learned_addr learned my addr 172.21.15.7:0/1137275011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/1137275011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0eaf50 0x5580ff0fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/1137275011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0f00c0 0x5580ff0f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c5f30640 1 -- 172.21.15.7:0/1137275011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0eaf50 msgr2=0x5580ff0fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/1137275011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0eaf50 0x5580ff0fba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c5f30640 1 -- 172.21.15.7:0/1137275011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0f00c0 msgr2=0x5580ff0f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/1137275011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0f00c0 0x5580ff0f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c5f30640 1 -- 172.21.15.7:0/1137275011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580fefeca80 con 0x5580ff0ea560 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/1137275011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0eaf50 0x5580ff0fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/1137275011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0e5eb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8002230 tx=0x7fb3b8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a77b5e248af92e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3aaffd640 1 -- 172.21.15.7:0/1137275011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8051690 con 0x5580ff0ea560 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3aaffd640 1 -- 172.21.15.7:0/1137275011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8032b20 con 0x5580ff0ea560 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.059+0000 7fb3aaffd640 1 -- 172.21.15.7:0/1137275011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8040700 con 0x5580ff0ea560 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 -- 172.21.15.7:0/1137275011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 msgr2=0x5580ff0e5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 --2- 172.21.15.7:0/1137275011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0e5eb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8002230 tx=0x7fb3b8005fd0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 -- 172.21.15.7:0/1137275011 shutdown_connections 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 --2- 172.21.15.7:0/1137275011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0f00c0 0x5580ff0f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 --2- 172.21.15.7:0/1137275011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0eaf50 0x5580ff0fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 --2- 172.21.15.7:0/1137275011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0ea560 0x5580ff0e5eb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 -- 172.21.15.7:0/1137275011 >> 172.21.15.7:0/1137275011 conn(0x5580ff0e5ad0 msgr2=0x5580ff0f24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 -- 172.21.15.7:0/1137275011 shutdown_connections 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 -- 172.21.15.7:0/1137275011 wait complete. 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 -- start start 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.060+0000 7fb3ca574740 1 Processor -- start 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0ea560 0x5580ff0f7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0f7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3ca574740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3ca574740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5580ff0faac0 con 0x5580ff0f00c0 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3ca574740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5580ff0fac30 con 0x5580ff0eaf50 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3ca574740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5580ff0fadf0 con 0x5580ff0ea560 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3ca574740 10 client.0 _ll_register_callbacks cb 0x7fb3b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c5f30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c5f30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34006/0 (socket says 172.21.15.7:34006) 2024-03-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c5f30640 1 -- 172.21.15.7:0/301834824 learned_addr learned my addr 172.21.15.7:0/301834824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0ea560 0x5580ff0f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c5f30640 1 -- 172.21.15.7:0/301834824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0ea560 msgr2=0x5580ff0f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0ea560 0x5580ff0f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c5f30640 1 -- 172.21.15.7:0/301834824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 msgr2=0x5580ff0f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c5f30640 1 -- 172.21.15.7:0/301834824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580ff050510 con 0x5580ff0f00c0 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3c5f30640 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0fa580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80096c0 tx=0x7fb3b8044040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b0ee2c31071a9b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/301834824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8003b70 con 0x5580ff0f00c0 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/301834824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb3b8003cd0 con 0x5580ff0f00c0 2024-03-23T13:19:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3ca574740 1 -- 172.21.15.7:0/301834824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b8034a90 con 0x5580ff0f00c0 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/301834824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3b8003e90 con 0x5580ff0f00c0 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3ca574740 10 client.15147 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.061+0000 7fb3ca574740 1 -- 172.21.15.7:0/301834824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580ff0127e0 con 0x5580ff0f00c0 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/301834824 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8034a90 con 0x5580ff0f00c0 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/301834824 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b8073080 con 0x5580ff0f00c0 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3ca574740 10 client.15147 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3ca574740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3ca574740 20 client.15147 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3ca574740 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5580ff16a810 0x5580ff169520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3ca574740 4 client.15147 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3ca574740 1 -- 172.21.15.7:0/301834824 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "status"}) -- 0x5580fed8e610 con 0x5580ff16a810 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5580ff16a810 0x5580ff169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3a8ff9640 10 client.15147 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.062+0000 7fb3c4f2e640 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5580ff16a810 0x5580ff169520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d7b3f7c974b14d server_cookie=acc457596976cac7 in_seq=0 out_seq=0 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3a8ff9640 1 -- 172.21.15.7:0/301834824 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5580fed8e610 con 0x5580ff16a810 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3a8ff9640 10 client.15147 handle_command_reply: tid=0 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 client.15147 shutdown 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 -- 172.21.15.7:0/301834824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 msgr2=0x5580ff0fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0fa580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80096c0 tx=0x7fb3b8044040 comp rx=0 tx=0).stop 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 -- 172.21.15.7:0/301834824 shutdown_connections 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5580ff16a810 0x5580ff169520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580ff0f00c0 0x5580ff0fa580 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5580ff0eaf50 0x5580ff0f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 --2- 172.21.15.7:0/301834824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5580ff0ea560 0x5580ff0f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3a8ff9640 0 client.15147 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 -- 172.21.15.7:0/301834824 >> 172.21.15.7:0/301834824 conn(0x5580ff0e5ad0 msgr2=0x5580ff0e5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 -- 172.21.15.7:0/301834824 shutdown_connections 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 1 -- 172.21.15.7:0/301834824 wait complete. 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3ca574740 20 client.15147 trim_cache size 0 max 16384 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24503, 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 88.541587901, 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-23T13:19:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T13:19:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 93.045412048000003 2024-03-23T13:19:10.072 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:19:10.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3c7993640 1 -- 172.21.15.7:0/950683279 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb3ac0a2150 msgr2=0x7fb3ac0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:10.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3c7993640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb3ac0a2150 0x7fb3ac0a4610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc09aab0 tx=0x7fb3bc06cdc0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3c7993640 1 -- 172.21.15.7:0/950683279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0100610 msgr2=0x7fb3c01977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3c7993640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0100610 0x7fb3c01977a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00e9c0 tx=0x7fb3bc00ee80 comp rx=0 tx=0).stop 2024-03-23T13:19:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3c7993640 1 -- 172.21.15.7:0/950683279 shutdown_connections 2024-03-23T13:19:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3c7993640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb3ac0a2150 0x7fb3ac0a4610 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3c7993640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3c01021c0 0x7fb3c019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3c7993640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3c0101800 0x7fb3c0197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3c7993640 1 --2- 172.21.15.7:0/950683279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0100610 0x7fb3c01977a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.063+0000 7fb3c7993640 1 -- 172.21.15.7:0/950683279 >> 172.21.15.7:0/950683279 conn(0x7fb3c00fbba0 msgr2=0x7fb3c00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.064+0000 7fb3c7993640 1 -- 172.21.15.7:0/950683279 shutdown_connections 2024-03-23T13:19:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:10.064+0000 7fb3c7993640 1 -- 172.21.15.7:0/950683279 wait complete. 2024-03-23T13:19:10.075 INFO:tasks.cephadm:Setup complete, yielding 2024-03-23T13:19:10.075 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T13:19:10.084 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-23T13:19:10.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-23T13:19:10.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:09 smithi130 ceph-mon[36225]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:19:10.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:09 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/4090573075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:10.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:09 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/4048920169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:19:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.030+0000 7f723bd6c700 1 -- 172.21.15.7:0/2247120840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7234103dd0 msgr2=0x7f723410a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.030+0000 7f723bd6c700 1 --2- 172.21.15.7:0/2247120840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7234103dd0 0x7f723410a680 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f722c00b0d0 tx=0x7f722c00b3e0 comp rx=0 tx=0).stop 2024-03-23T13:19:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.030+0000 7f723bd6c700 1 -- 172.21.15.7:0/2247120840 shutdown_connections 2024-03-23T13:19:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.030+0000 7f723bd6c700 1 --2- 172.21.15.7:0/2247120840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7234103dd0 0x7f723410a680 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.030+0000 7f723bd6c700 1 --2- 172.21.15.7:0/2247120840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7234102230 0x7f7234102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.030+0000 7f723bd6c700 1 --2- 172.21.15.7:0/2247120840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7234103430 0x7f7234103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.030+0000 7f723bd6c700 1 -- 172.21.15.7:0/2247120840 >> 172.21.15.7:0/2247120840 conn(0x7f72340fd810 msgr2=0x7f72340ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.030+0000 7f723bd6c700 1 -- 172.21.15.7:0/2247120840 shutdown_connections 2024-03-23T13:19:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.030+0000 7f723bd6c700 1 -- 172.21.15.7:0/2247120840 wait complete. 2024-03-23T13:19:12.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:11 smithi007 ceph-mon[32184]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:19:12.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.030+0000 7f723bd6c700 1 Processor -- start 2024-03-23T13:19:12.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723bd6c700 1 -- start start 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723bd6c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7234102230 0x7f723419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723bd6c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7234103430 0x7f723419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723bd6c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7234103dd0 0x7f72341a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723bd6c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72341a20a0 con 0x7f7234103dd0 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723bd6c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f72341a21e0 con 0x7f7234103430 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723bd6c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72341a2370 con 0x7f7234102230 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723a309700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7234103dd0 0x7f72341a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f7239b08700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7234102230 0x7f723419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f7239b08700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7234102230 0x7f723419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:36760/0 (socket says 172.21.15.7:36760) 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f7239b08700 1 -- 172.21.15.7:0/2115902489 learned_addr learned my addr 172.21.15.7:0/2115902489 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723a309700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7234103dd0 0x7f72341a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34028/0 (socket says 172.21.15.7:34028) 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f7239307700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7234103430 0x7f723419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723a309700 1 -- 172.21.15.7:0/2115902489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7234102230 msgr2=0x7f723419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723a309700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7234102230 0x7f723419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723a309700 1 -- 172.21.15.7:0/2115902489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7234103430 msgr2=0x7f723419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723a309700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7234103430 0x7f723419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723a309700 1 -- 172.21.15.7:0/2115902489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722c009d00 con 0x7f7234103dd0 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f7239b08700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7234102230 0x7f723419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.031+0000 7f723a309700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7234103dd0 0x7f72341a1ad0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f722c00b820 tx=0x7f722c009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.032+0000 7f722affd700 1 -- 172.21.15.7:0/2115902489 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c00e040 con 0x7f7234103dd0 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.032+0000 7f722affd700 1 -- 172.21.15.7:0/2115902489 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f722c003e10 con 0x7f7234103dd0 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.032+0000 7f722affd700 1 -- 172.21.15.7:0/2115902489 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c01d9f0 con 0x7f7234103dd0 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.032+0000 7f723bd6c700 1 -- 172.21.15.7:0/2115902489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72341a25f0 con 0x7f7234103dd0 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.032+0000 7f723bd6c700 1 -- 172.21.15.7:0/2115902489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72341a2a60 con 0x7f7234103dd0 2024-03-23T13:19:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.033+0000 7f723bd6c700 1 -- 172.21.15.7:0/2115902489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7234068ec0 con 0x7f7234103dd0 2024-03-23T13:19:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.034+0000 7f722affd700 1 -- 172.21.15.7:0/2115902489 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 136806+0+0 (secure 0 0 0) 0x7f722c005670 con 0x7f7234103dd0 2024-03-23T13:19:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.034+0000 7f722affd700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72200a2260 0x7f72200a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.035+0000 7f7239b08700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72200a2260 0x7f72200a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.035+0000 7f722affd700 1 -- 172.21.15.7:0/2115902489 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f722c0cef60 con 0x7f7234103dd0 2024-03-23T13:19:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.035+0000 7f7239b08700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72200a2260 0x7f72200a4720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7230007fd0 tx=0x7f7230007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.038+0000 7f722affd700 1 -- 172.21.15.7:0/2115902489 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f722c097f00 con 0x7f7234103dd0 2024-03-23T13:19:12.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:11 smithi155 ceph-mon[36157]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:19:12.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:11 smithi130 ceph-mon[36225]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:19:12.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.259+0000 7f723bd6c700 1 -- 172.21.15.7:0/2115902489 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7234075b50 con 0x7f72200a2260 2024-03-23T13:19:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.260+0000 7f722affd700 1 -- 172.21.15.7:0/2115902489 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7234075b50 con 0x7f72200a2260 2024-03-23T13:19:12.260 INFO:teuthology.orchestra.run.smithi007.stdout:Backend: cephadm 2024-03-23T13:19:12.260 INFO:teuthology.orchestra.run.smithi007.stdout:Available: Yes 2024-03-23T13:19:12.260 INFO:teuthology.orchestra.run.smithi007.stdout:Paused: No 2024-03-23T13:19:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 -- 172.21.15.7:0/2115902489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72200a2260 msgr2=0x7f72200a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72200a2260 0x7f72200a4720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7230007fd0 tx=0x7f7230007480 comp rx=0 tx=0).stop 2024-03-23T13:19:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 -- 172.21.15.7:0/2115902489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7234103dd0 msgr2=0x7f72341a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7234103dd0 0x7f72341a1ad0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f722c00b820 tx=0x7f722c009380 comp rx=0 tx=0).stop 2024-03-23T13:19:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 -- 172.21.15.7:0/2115902489 shutdown_connections 2024-03-23T13:19:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f72200a2260 0x7f72200a4720 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7234103dd0 0x7f72341a1ad0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7234102230 0x7f723419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 --2- 172.21.15.7:0/2115902489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7234103430 0x7f723419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 -- 172.21.15.7:0/2115902489 >> 172.21.15.7:0/2115902489 conn(0x7f72340fd810 msgr2=0x7f7234074430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 -- 172.21.15.7:0/2115902489 shutdown_connections 2024-03-23T13:19:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:12.262+0000 7f723bd6c700 1 -- 172.21.15.7:0/2115902489 wait complete. 2024-03-23T13:19:12.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-23T13:19:13.217 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:19:14.038 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:13 smithi155 ceph-mon[36157]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:19:14.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:13 smithi155 ceph-mon[36157]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:14.053 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:13 smithi007 ceph-mon[32184]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:19:14.053 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:13 smithi007 ceph-mon[32184]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:14.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:13 smithi130 ceph-mon[36225]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:19:14.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:13 smithi130 ceph-mon[36225]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.899+0000 7ff19cf7c700 1 -- 172.21.15.7:0/1853805991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198101130 msgr2=0x7ff198101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.899+0000 7ff19cf7c700 1 --2- 172.21.15.7:0/1853805991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198101130 0x7ff198101550 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff1880099c0 tx=0x7ff188009cd0 comp rx=0 tx=0).stop 2024-03-23T13:19:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.899+0000 7ff19cf7c700 1 -- 172.21.15.7:0/1853805991 shutdown_connections 2024-03-23T13:19:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.899+0000 7ff19cf7c700 1 --2- 172.21.15.7:0/1853805991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff198102cd0 0x7ff198109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.899+0000 7ff19cf7c700 1 --2- 172.21.15.7:0/1853805991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198101130 0x7ff198101550 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.899+0000 7ff19cf7c700 1 --2- 172.21.15.7:0/1853805991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff198102330 0x7ff198102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.899+0000 7ff19cf7c700 1 -- 172.21.15.7:0/1853805991 >> 172.21.15.7:0/1853805991 conn(0x7ff1980fc6d0 msgr2=0x7ff1980feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.899+0000 7ff19cf7c700 1 -- 172.21.15.7:0/1853805991 shutdown_connections 2024-03-23T13:19:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.899+0000 7ff19cf7c700 1 -- 172.21.15.7:0/1853805991 wait complete. 2024-03-23T13:19:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff19cf7c700 1 Processor -- start 2024-03-23T13:19:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff19cf7c700 1 -- start start 2024-03-23T13:19:14.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff19cf7c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff198101130 0x7ff198197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:14.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff19cf7c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff198102330 0x7ff198197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff19cf7c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198102cd0 0x7ff19819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff19cf7c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff19819ecc0 con 0x7ff198102cd0 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff19cf7c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff19819ee00 con 0x7ff198102330 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff19cf7c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff19819ef90 con 0x7ff198101130 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff196d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198102cd0 0x7ff19819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff196d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198102cd0 0x7ff19819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52834/0 (socket says 172.21.15.7:52834) 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff196d9d700 1 -- 172.21.15.7:0/3399479371 learned_addr learned my addr 172.21.15.7:0/3399479371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff195d9b700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff198102330 0x7ff198197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.900+0000 7ff19659c700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff198101130 0x7ff198197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff195d9b700 1 -- 172.21.15.7:0/3399479371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff198101130 msgr2=0x7ff198197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff195d9b700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff198101130 0x7ff198197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff195d9b700 1 -- 172.21.15.7:0/3399479371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198102cd0 msgr2=0x7ff19819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff195d9b700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198102cd0 0x7ff19819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff195d9b700 1 -- 172.21.15.7:0/3399479371 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1880095f0 con 0x7ff198102330 2024-03-23T13:19:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff19659c700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff198101130 0x7ff198197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff196d9d700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198102cd0 0x7ff19819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff195d9b700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff198102330 0x7ff198197670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff1800078c0 tx=0x7ff180007bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:14.904 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:14 smithi007 ceph-mon[32184]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-23T13:19:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff18f7fe700 1 -- 172.21.15.7:0/3399479371 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff180012070 con 0x7ff198102330 2024-03-23T13:19:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff18f7fe700 1 -- 172.21.15.7:0/3399479371 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff180004500 con 0x7ff198102330 2024-03-23T13:19:14.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff18f7fe700 1 -- 172.21.15.7:0/3399479371 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1800054c0 con 0x7ff198102330 2024-03-23T13:19:14.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff19cf7c700 1 -- 172.21.15.7:0/3399479371 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff198069f10 con 0x7ff198102330 2024-03-23T13:19:14.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.901+0000 7ff19cf7c700 1 -- 172.21.15.7:0/3399479371 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19806a3c0 con 0x7ff198102330 2024-03-23T13:19:14.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.902+0000 7ff19cf7c700 1 -- 172.21.15.7:0/3399479371 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff198068ec0 con 0x7ff198102330 2024-03-23T13:19:14.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.904+0000 7ff18f7fe700 1 -- 172.21.15.7:0/3399479371 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7ff180016070 con 0x7ff198102330 2024-03-23T13:19:14.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.904+0000 7ff18f7fe700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff1840a21d0 0x7ff1840a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:14.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.905+0000 7ff19659c700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff1840a21d0 0x7ff1840a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:14.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.905+0000 7ff18f7fe700 1 -- 172.21.15.7:0/3399479371 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff1800cea20 con 0x7ff198102330 2024-03-23T13:19:14.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.905+0000 7ff19659c700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff1840a21d0 0x7ff1840a4690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff188006010 tx=0x7ff188000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:14.909+0000 7ff18f7fe700 1 -- 172.21.15.7:0/3399479371 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff18009b530 con 0x7ff198102330 2024-03-23T13:19:15.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.122+0000 7ff19cf7c700 1 -- 172.21.15.7:0/3399479371 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff198108050 con 0x7ff1840a21d0 2024-03-23T13:19:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.136+0000 7ff18f7fe700 1 -- 172.21.15.7:0/3399479371 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7ff198108050 con 0x7ff1840a21d0 2024-03-23T13:19:15.137 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T13:19:15.137 INFO:teuthology.orchestra.run.smithi007.stdout:mds.a smithi007 running (101s) 16s ago 100s 18.6M - 19.0.0-1855-g21b3bd67 553463a90fd5 256ad5530e33 2024-03-23T13:19:15.137 INFO:teuthology.orchestra.run.smithi007.stdout:mds.b smithi130 running (98s) 31s ago 98s 25.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 4fde6ee0de3a 2024-03-23T13:19:15.137 INFO:teuthology.orchestra.run.smithi007.stdout:mds.c smithi155 running (112s) 31s ago 112s 19.0M - 19.0.0-1855-g21b3bd67 553463a90fd5 17af4a18e4c4 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mds.d smithi007 running (2m) 16s ago 2m 13.8M - 19.0.0-1855-g21b3bd67 553463a90fd5 d1feeb764dee 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mds.e smithi130 running (118s) 31s ago 117s 17.6M - 19.0.0-1855-g21b3bd67 553463a90fd5 c0da684a2129 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mds.f smithi155 running (115s) 31s ago 115s 18.6M - 19.0.0-1855-g21b3bd67 553463a90fd5 952c73e93ab4 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mds.g smithi007 running (104s) 16s ago 103s 18.6M - 19.0.0-1855-g21b3bd67 553463a90fd5 b0f5d7404803 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mds.h smithi130 running (109s) 31s ago 109s 19.7M - 19.0.0-1855-g21b3bd67 553463a90fd5 189dca2e132b 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mds.i smithi155 running (107s) 31s ago 106s 17.0M - 19.0.0-1855-g21b3bd67 553463a90fd5 255fec02aba7 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mds.j smithi007 running (2m) 16s ago 2m 16.1M - 19.0.0-1855-g21b3bd67 553463a90fd5 287fc6b1b0f5 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mds.k smithi130 running (2m) 31s ago 2m 19.1M - 19.0.0-1855-g21b3bd67 553463a90fd5 0c546a506d7e 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mds.l smithi155 running (2m) 31s ago 2m 16.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 c3a25fe4bd6d 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.x smithi007 *:9283,8765 running (9m) 16s ago 9m 512M - 19.0.0-1855-g21b3bd67 553463a90fd5 67d150695aee 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.y smithi130 *:8443,8765 running (7m) 31s ago 7m 452M - 19.0.0-1855-g21b3bd67 553463a90fd5 0b3e73965526 2024-03-23T13:19:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.z smithi155 *:8443,8765 running (7m) 31s ago 7m 453M - 19.0.0-1855-g21b3bd67 553463a90fd5 3499c3558194 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a smithi007 running (9m) 16s ago 9m 53.4M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 d3f413d67b18 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b smithi130 running (7m) 31s ago 7m 39.5M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 42a461ed95f7 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c smithi155 running (7m) 31s ago 7m 38.7M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 7ab6cf0d7485 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (7m) 16s ago 7m 44.4M 4273M 19.0.0-1855-g21b3bd67 553463a90fd5 e61efa28d77b 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi130 running (6m) 31s ago 6m 56.2M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 2451974fed84 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi155 running (6m) 31s ago 6m 56.6M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 4e675c037796 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi007 running (5m) 16s ago 5m 52.7M 4273M 19.0.0-1855-g21b3bd67 553463a90fd5 ae287f814aa9 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi130 running (5m) 31s ago 5m 43.1M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 dbb5c5e85203 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi155 running (5m) 31s ago 5m 43.3M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 96021899651f 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 smithi007 running (4m) 16s ago 4m 51.7M 4273M 19.0.0-1855-g21b3bd67 553463a90fd5 0c3a434d12f7 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 smithi130 running (4m) 31s ago 4m 57.2M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 37eab996ac17 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 smithi155 running (3m) 31s ago 3m 41.9M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 3200724919af 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 smithi007 running (3m) 16s ago 3m 52.9M 4273M 19.0.0-1855-g21b3bd67 553463a90fd5 bf2e9ff3f6c4 2024-03-23T13:19:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 smithi130 running (2m) 31s ago 2m 47.2M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 b5373872d659 2024-03-23T13:19:15.140 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 smithi155 running (2m) 31s ago 2m 40.6M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 bd56daa819f5 2024-03-23T13:19:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 -- 172.21.15.7:0/3399479371 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff1840a21d0 msgr2=0x7ff1840a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff1840a21d0 0x7ff1840a4690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff188006010 tx=0x7ff188000bc0 comp rx=0 tx=0).stop 2024-03-23T13:19:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 -- 172.21.15.7:0/3399479371 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff198102330 msgr2=0x7ff198197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff198102330 0x7ff198197670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff1800078c0 tx=0x7ff180007bd0 comp rx=0 tx=0).stop 2024-03-23T13:19:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 -- 172.21.15.7:0/3399479371 shutdown_connections 2024-03-23T13:19:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff1840a21d0 0x7ff1840a4690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198102cd0 0x7ff19819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff198101130 0x7ff198197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 --2- 172.21.15.7:0/3399479371 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff198102330 0x7ff198197670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 -- 172.21.15.7:0/3399479371 >> 172.21.15.7:0/3399479371 conn(0x7ff1980fc6d0 msgr2=0x7ff1981056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 -- 172.21.15.7:0/3399479371 shutdown_connections 2024-03-23T13:19:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:15.138+0000 7ff19cf7c700 1 -- 172.21.15.7:0/3399479371 wait complete. 2024-03-23T13:19:15.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:14 smithi130 ceph-mon[36225]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-23T13:19:15.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:14 smithi155 ceph-mon[36157]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-23T13:19:15.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-23T13:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:19:16.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:15 smithi007 ceph-mon[32184]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T13:19:16.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:15 smithi130 ceph-mon[36225]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:15 smithi155 ceph-mon[36157]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T13:19:17.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:16 smithi007 ceph-mon[32184]: from='client.24980 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:17.190 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:16 smithi155 ceph-mon[36157]: from='client.24980 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:17.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:16 smithi130 ceph-mon[36225]: from='client.24980 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.700+0000 7f7743e20700 1 -- 172.21.15.7:0/898469027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c102d30 msgr2=0x7f773c105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.700+0000 7f7743e20700 1 --2- 172.21.15.7:0/898469027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c102d30 0x7f773c105120 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f773400b0a0 tx=0x7f773400b3b0 comp rx=0 tx=0).stop 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.700+0000 7f7743e20700 1 -- 172.21.15.7:0/898469027 shutdown_connections 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.700+0000 7f7743e20700 1 --2- 172.21.15.7:0/898469027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f773c107f90 0x7f773c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.700+0000 7f7743e20700 1 --2- 172.21.15.7:0/898469027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c102d30 0x7f773c105120 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.700+0000 7f7743e20700 1 --2- 172.21.15.7:0/898469027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773c105660 0x7f773c107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.700+0000 7f7743e20700 1 -- 172.21.15.7:0/898469027 >> 172.21.15.7:0/898469027 conn(0x7f773c0fc760 msgr2=0x7f773c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.700+0000 7f7743e20700 1 -- 172.21.15.7:0/898469027 shutdown_connections 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.700+0000 7f7743e20700 1 -- 172.21.15.7:0/898469027 wait complete. 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7743e20700 1 Processor -- start 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7743e20700 1 -- start start 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7743e20700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773c102d30 0x7f773c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7743e20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c105660 0x7f773c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7743e20700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f773c107f90 0x7f773c19f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7743e20700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f773c19ff00 con 0x7f773c105660 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7743e20700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f773c1a0040 con 0x7f773c107f90 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7743e20700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f773c1a01d0 con 0x7f773c102d30 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f77413bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c105660 0x7f773c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7741bbc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773c102d30 0x7f773c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7741bbc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773c102d30 0x7f773c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:42628/0 (socket says 172.21.15.7:42628) 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f77413bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c105660 0x7f773c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52858/0 (socket says 172.21.15.7:52858) 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7741bbc700 1 -- 172.21.15.7:0/1778428080 learned_addr learned my addr 172.21.15.7:0/1778428080 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f77423bd700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f773c107f90 0x7f773c19f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f77413bb700 1 -- 172.21.15.7:0/1778428080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773c102d30 msgr2=0x7f773c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f77413bb700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773c102d30 0x7f773c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f77413bb700 1 -- 172.21.15.7:0/1778428080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f773c107f90 msgr2=0x7f773c19f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f77413bb700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f773c107f90 0x7f773c19f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f77413bb700 1 -- 172.21.15.7:0/1778428080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7734009d00 con 0x7f773c105660 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f7741bbc700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773c102d30 0x7f773c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.701+0000 7f77413bb700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c105660 0x7f773c1988b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f773000e810 tx=0x7f773000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.702+0000 7f773affd700 1 -- 172.21.15.7:0/1778428080 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730004220 con 0x7f773c105660 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.702+0000 7f7743e20700 1 -- 172.21.15.7:0/1778428080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773c1a04b0 con 0x7f773c105660 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.702+0000 7f773affd700 1 -- 172.21.15.7:0/1778428080 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7730004d10 con 0x7f773c105660 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.702+0000 7f773affd700 1 -- 172.21.15.7:0/1778428080 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730005710 con 0x7f773c105660 2024-03-23T13:19:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.702+0000 7f7743e20700 1 -- 172.21.15.7:0/1778428080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773c1a09b0 con 0x7f773c105660 2024-03-23T13:19:17.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.703+0000 7f7743e20700 1 -- 172.21.15.7:0/1778428080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773c0fe2e0 con 0x7f773c105660 2024-03-23T13:19:17.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.704+0000 7f773affd700 1 -- 172.21.15.7:0/1778428080 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f7730004750 con 0x7f773c105660 2024-03-23T13:19:17.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.705+0000 7f773affd700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77240a2150 0x7f77240a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:17.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.705+0000 7f7741bbc700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77240a2150 0x7f77240a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:17.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.705+0000 7f773affd700 1 -- 172.21.15.7:0/1778428080 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7730014070 con 0x7f773c105660 2024-03-23T13:19:17.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.705+0000 7f7741bbc700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77240a2150 0x7f77240a4610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7734007ab0 tx=0x7f773400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:17.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.710+0000 7f773affd700 1 -- 172.21.15.7:0/1778428080 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7730096990 con 0x7f773c105660 2024-03-23T13:19:17.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.921+0000 7f7743e20700 1 -- 172.21.15.7:0/1778428080 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f773c02db60 con 0x7f77240a2150 2024-03-23T13:19:17.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.927+0000 7f773affd700 1 -- 172.21.15.7:0/1778428080 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f773c02db60 con 0x7f77240a2150 2024-03-23T13:19:17.928 INFO:teuthology.orchestra.run.smithi007.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-23T13:19:17.928 INFO:teuthology.orchestra.run.smithi007.stdout:mds.all 12/12 34s ago 2m smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi130=b;smithi130=e;smithi130=h;smithi130=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-03-23T13:19:17.928 INFO:teuthology.orchestra.run.smithi007.stdout:mgr 3/3 34s ago 7m smithi007=x;smithi130=y;smithi155=z;count:3 2024-03-23T13:19:17.928 INFO:teuthology.orchestra.run.smithi007.stdout:mon 3/3 34s ago 7m smithi007:172.21.15.7=a;smithi130:172.21.15.130=b;smithi155:172.21.15.155=c;count:3 2024-03-23T13:19:17.928 INFO:teuthology.orchestra.run.smithi007.stdout:osd 12 34s ago - 2024-03-23T13:19:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 -- 172.21.15.7:0/1778428080 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77240a2150 msgr2=0x7f77240a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77240a2150 0x7f77240a4610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7734007ab0 tx=0x7f773400b700 comp rx=0 tx=0).stop 2024-03-23T13:19:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 -- 172.21.15.7:0/1778428080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c105660 msgr2=0x7f773c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c105660 0x7f773c1988b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f773000e810 tx=0x7f773000eb20 comp rx=0 tx=0).stop 2024-03-23T13:19:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 -- 172.21.15.7:0/1778428080 shutdown_connections 2024-03-23T13:19:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77240a2150 0x7f77240a4610 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f773c107f90 0x7f773c19f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773c102d30 0x7f773c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 --2- 172.21.15.7:0/1778428080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c105660 0x7f773c1988b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 -- 172.21.15.7:0/1778428080 >> 172.21.15.7:0/1778428080 conn(0x7f773c0fc760 msgr2=0x7f773c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.929+0000 7f7743e20700 1 -- 172.21.15.7:0/1778428080 shutdown_connections 2024-03-23T13:19:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:17.930+0000 7f7743e20700 1 -- 172.21.15.7:0/1778428080 wait complete. 2024-03-23T13:19:18.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:17 smithi007 ceph-mon[32184]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-23T13:19:18.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:17 smithi130 ceph-mon[36225]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-23T13:19:18.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:17 smithi155 ceph-mon[36157]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-23T13:19:18.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-23T13:19:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:19:20.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:19 smithi007 ceph-mon[32184]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:20.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:19 smithi007 ceph-mon[32184]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-23T13:19:20.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:19 smithi130 ceph-mon[36225]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:20.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:19 smithi130 ceph-mon[36225]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-23T13:19:20.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:19 smithi155 ceph-mon[36157]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:20.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:19 smithi155 ceph-mon[36157]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-23T13:19:20.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.797+0000 7f011a761700 1 -- 172.21.15.7:0/2145678612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0114102230 msgr2=0x7f0114102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:20.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.797+0000 7f011a761700 1 --2- 172.21.15.7:0/2145678612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0114102230 0x7f0114102650 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f01040099c0 tx=0x7f0104009cd0 comp rx=0 tx=0).stop 2024-03-23T13:19:20.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.797+0000 7f011a761700 1 -- 172.21.15.7:0/2145678612 shutdown_connections 2024-03-23T13:19:20.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.797+0000 7f011a761700 1 --2- 172.21.15.7:0/2145678612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0114103dd0 0x7f011410a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.797+0000 7f011a761700 1 --2- 172.21.15.7:0/2145678612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0114102230 0x7f0114102650 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.797+0000 7f011a761700 1 --2- 172.21.15.7:0/2145678612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0114103430 0x7f0114103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.797+0000 7f011a761700 1 -- 172.21.15.7:0/2145678612 >> 172.21.15.7:0/2145678612 conn(0x7f01140fd810 msgr2=0x7f01140ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.797+0000 7f011a761700 1 -- 172.21.15.7:0/2145678612 shutdown_connections 2024-03-23T13:19:20.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.797+0000 7f011a761700 1 -- 172.21.15.7:0/2145678612 wait complete. 2024-03-23T13:19:20.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.798+0000 7f011a761700 1 Processor -- start 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.798+0000 7f011a761700 1 -- start start 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f011a761700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0114102230 0x7f01141981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f011a761700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0114103430 0x7f0114198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f011a761700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0114103dd0 0x7f011419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f011a761700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f011419fd80 con 0x7f0114103dd0 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f011a761700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f011419fec0 con 0x7f0114103430 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f011a761700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01141a0050 con 0x7f0114102230 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f0118cfe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0114103dd0 0x7f011419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f0118cfe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0114103dd0 0x7f011419f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52884/0 (socket says 172.21.15.7:52884) 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f0118cfe700 1 -- 172.21.15.7:0/3464260195 learned_addr learned my addr 172.21.15.7:0/3464260195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f0113fff700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0114102230 0x7f01141981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f01137fe700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0114103430 0x7f0114198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f0118cfe700 1 -- 172.21.15.7:0/3464260195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0114102230 msgr2=0x7f01141981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:20.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f0118cfe700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0114102230 0x7f01141981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f0118cfe700 1 -- 172.21.15.7:0/3464260195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0114103430 msgr2=0x7f0114198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f0118cfe700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0114103430 0x7f0114198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f0118cfe700 1 -- 172.21.15.7:0/3464260195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01040095f0 con 0x7f0114103dd0 2024-03-23T13:19:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f01137fe700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0114103430 0x7f0114198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f0118cfe700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0114103dd0 0x7f011419f7b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f010c00ef60 tx=0x7f010c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f01117fa700 1 -- 172.21.15.7:0/3464260195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c018070 con 0x7f0114103dd0 2024-03-23T13:19:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f01117fa700 1 -- 172.21.15.7:0/3464260195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c00f040 con 0x7f0114103dd0 2024-03-23T13:19:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.799+0000 7f011a761700 1 -- 172.21.15.7:0/3464260195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01141a02d0 con 0x7f0114103dd0 2024-03-23T13:19:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.800+0000 7f01117fa700 1 -- 172.21.15.7:0/3464260195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c00cd60 con 0x7f0114103dd0 2024-03-23T13:19:20.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.800+0000 7f011a761700 1 -- 172.21.15.7:0/3464260195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01141a0780 con 0x7f0114103dd0 2024-03-23T13:19:20.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.800+0000 7f011a761700 1 -- 172.21.15.7:0/3464260195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0114109670 con 0x7f0114103dd0 2024-03-23T13:19:20.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.802+0000 7f01117fa700 1 -- 172.21.15.7:0/3464260195 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f010c004ad0 con 0x7f0114103dd0 2024-03-23T13:19:20.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.802+0000 7f01117fa700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01000a2150 0x7f01000a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:20.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.803+0000 7f0113fff700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01000a2150 0x7f01000a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:20.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.803+0000 7f01117fa700 1 -- 172.21.15.7:0/3464260195 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f010c0cde00 con 0x7f0114103dd0 2024-03-23T13:19:20.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.803+0000 7f0113fff700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01000a2150 0x7f01000a4610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01040036c0 tx=0x7f0104005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:20.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:20.808+0000 7f01117fa700 1 -- 172.21.15.7:0/3464260195 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f010c096e10 con 0x7f0114103dd0 2024-03-23T13:19:21.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.017+0000 7f011a761700 1 -- 172.21.15.7:0/3464260195 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f011402db60 con 0x7f01000a2150 2024-03-23T13:19:21.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.019+0000 7f01117fa700 1 -- 172.21.15.7:0/3464260195 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f011402db60 con 0x7f01000a2150 2024-03-23T13:19:21.019 INFO:teuthology.orchestra.run.smithi007.stdout:HOST ADDR LABELS STATUS 2024-03-23T13:19:21.019 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 172.21.15.7 2024-03-23T13:19:21.019 INFO:teuthology.orchestra.run.smithi007.stdout:smithi130 172.21.15.130 2024-03-23T13:19:21.019 INFO:teuthology.orchestra.run.smithi007.stdout:smithi155 172.21.15.155 2024-03-23T13:19:21.020 INFO:teuthology.orchestra.run.smithi007.stdout:3 hosts in cluster 2024-03-23T13:19:21.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 -- 172.21.15.7:0/3464260195 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01000a2150 msgr2=0x7f01000a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01000a2150 0x7f01000a4610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01040036c0 tx=0x7f0104005ad0 comp rx=0 tx=0).stop 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 -- 172.21.15.7:0/3464260195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0114103dd0 msgr2=0x7f011419f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0114103dd0 0x7f011419f7b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f010c00ef60 tx=0x7f010c00ef90 comp rx=0 tx=0).stop 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 -- 172.21.15.7:0/3464260195 shutdown_connections 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01000a2150 0x7f01000a4610 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0114103dd0 0x7f011419f7b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0114102230 0x7f01141981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 --2- 172.21.15.7:0/3464260195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0114103430 0x7f0114198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 -- 172.21.15.7:0/3464260195 >> 172.21.15.7:0/3464260195 conn(0x7f01140fd810 msgr2=0x7f01141067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 -- 172.21.15.7:0/3464260195 shutdown_connections 2024-03-23T13:19:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:21.021+0000 7f011a761700 1 -- 172.21.15.7:0/3464260195 wait complete. 2024-03-23T13:19:21.788 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-23T13:19:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:19:22.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:21 smithi007 ceph-mon[32184]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-23T13:19:22.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:21 smithi130 ceph-mon[36225]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-23T13:19:22.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:21 smithi155 ceph-mon[36157]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-23T13:19:23.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:22 smithi007 ceph-mon[32184]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:23.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:22 smithi130 ceph-mon[36225]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:23.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:22 smithi155 ceph-mon[36157]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.849+0000 7fc108f41700 1 -- 172.21.15.7:0/117748851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104102230 msgr2=0x7fc104102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.849+0000 7fc108f41700 1 --2- 172.21.15.7:0/117748851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104102230 0x7fc104102650 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40099c0 tx=0x7fc0f4009cd0 comp rx=0 tx=0).stop 2024-03-23T13:19:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.849+0000 7fc108f41700 1 -- 172.21.15.7:0/117748851 shutdown_connections 2024-03-23T13:19:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.849+0000 7fc108f41700 1 --2- 172.21.15.7:0/117748851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc104103dd0 0x7fc10410a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.849+0000 7fc108f41700 1 --2- 172.21.15.7:0/117748851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104102230 0x7fc104102650 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.849+0000 7fc108f41700 1 --2- 172.21.15.7:0/117748851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104103430 0x7fc104103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.849+0000 7fc108f41700 1 -- 172.21.15.7:0/117748851 >> 172.21.15.7:0/117748851 conn(0x7fc1040fd810 msgr2=0x7fc1040ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.849+0000 7fc108f41700 1 -- 172.21.15.7:0/117748851 shutdown_connections 2024-03-23T13:19:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.849+0000 7fc108f41700 1 -- 172.21.15.7:0/117748851 wait complete. 2024-03-23T13:19:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.850+0000 7fc108f41700 1 Processor -- start 2024-03-23T13:19:23.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.850+0000 7fc108f41700 1 -- start start 2024-03-23T13:19:23.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:23 smithi007 ceph-mon[32184]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T13:19:23.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.850+0000 7fc108f41700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc104102230 0x7fc1041981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:23.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.850+0000 7fc108f41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104103430 0x7fc104198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.850+0000 7fc108f41700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104103dd0 0x7fc10419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.850+0000 7fc108f41700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc10419fd70 con 0x7fc104103430 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.850+0000 7fc108f41700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc10419feb0 con 0x7fc104103dd0 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.850+0000 7fc108f41700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1041a0040 con 0x7fc104102230 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc10259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc104102230 0x7fc1041981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc10259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc104102230 0x7fc1041981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:42662/0 (socket says 172.21.15.7:42662) 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc10259c700 1 -- 172.21.15.7:0/2022707853 learned_addr learned my addr 172.21.15.7:0/2022707853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc101d9b700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104103430 0x7fc104198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc102d9d700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104103dd0 0x7fc10419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc10259c700 1 -- 172.21.15.7:0/2022707853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104103dd0 msgr2=0x7fc10419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc10259c700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104103dd0 0x7fc10419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc10259c700 1 -- 172.21.15.7:0/2022707853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104103430 msgr2=0x7fc104198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc10259c700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104103430 0x7fc104198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc10259c700 1 -- 172.21.15.7:0/2022707853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f40095f0 con 0x7fc104102230 2024-03-23T13:19:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc102d9d700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104103dd0 0x7fc10419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:19:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc10259c700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc104102230 0x7fc1041981e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009fd0 tx=0x7fc0f400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc0f37fe700 1 -- 172.21.15.7:0/2022707853 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f401d070 con 0x7fc104102230 2024-03-23T13:19:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc0f37fe700 1 -- 172.21.15.7:0/2022707853 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0f4005940 con 0x7fc104102230 2024-03-23T13:19:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc108f41700 1 -- 172.21.15.7:0/2022707853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc104074570 con 0x7fc104102230 2024-03-23T13:19:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc0f37fe700 1 -- 172.21.15.7:0/2022707853 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40173f0 con 0x7fc104102230 2024-03-23T13:19:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.851+0000 7fc108f41700 1 -- 172.21.15.7:0/2022707853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc104074a60 con 0x7fc104102230 2024-03-23T13:19:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.852+0000 7fc108f41700 1 -- 172.21.15.7:0/2022707853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1041095f0 con 0x7fc104102230 2024-03-23T13:19:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.853+0000 7fc0f37fe700 1 -- 172.21.15.7:0/2022707853 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fc0f400fd20 con 0x7fc104102230 2024-03-23T13:19:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.854+0000 7fc0f37fe700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0ec0a21d0 0x7fc0ec0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.854+0000 7fc0f37fe700 1 -- 172.21.15.7:0/2022707853 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc0f40cea30 con 0x7fc104102230 2024-03-23T13:19:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.854+0000 7fc101d9b700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0ec0a21d0 0x7fc0ec0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.854+0000 7fc101d9b700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0ec0a21d0 0x7fc0ec0a4690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009c60 tx=0x7fc0f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:23.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:23.858+0000 7fc0f37fe700 1 -- 172.21.15.7:0/2022707853 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc0f4097aa0 con 0x7fc104102230 2024-03-23T13:19:24.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.069+0000 7fc108f41700 1 -- 172.21.15.7:0/2022707853 --> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc10402db60 con 0x7fc0ec0a21d0 2024-03-23T13:19:24.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.070+0000 7fc0f37fe700 1 -- 172.21.15.7:0/2022707853 <== mgr.14152 v2:172.21.15.7:6800/1535208045 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc10402db60 con 0x7fc0ec0a21d0 2024-03-23T13:19:24.071 INFO:teuthology.orchestra.run.smithi007.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-23T13:19:24.071 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GE480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-23T13:19:24.071 INFO:teuthology.orchestra.run.smithi007.stdout:smithi130 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500Q1400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-23T13:19:24.071 INFO:teuthology.orchestra.run.smithi007.stdout:smithi155 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GW400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 -- 172.21.15.7:0/2022707853 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0ec0a21d0 msgr2=0x7fc0ec0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0ec0a21d0 0x7fc0ec0a4690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009c60 tx=0x7fc0f8009260 comp rx=0 tx=0).stop 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 -- 172.21.15.7:0/2022707853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc104102230 msgr2=0x7fc1041981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc104102230 0x7fc1041981e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009fd0 tx=0x7fc0f400f690 comp rx=0 tx=0).stop 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 -- 172.21.15.7:0/2022707853 shutdown_connections 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc0ec0a21d0 0x7fc0ec0a4690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc104103dd0 0x7fc10419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc104102230 0x7fc1041981e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 --2- 172.21.15.7:0/2022707853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc104103430 0x7fc104198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 -- 172.21.15.7:0/2022707853 >> 172.21.15.7:0/2022707853 conn(0x7fc1040fd810 msgr2=0x7fc104106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 -- 172.21.15.7:0/2022707853 shutdown_connections 2024-03-23T13:19:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:24.073+0000 7fc108f41700 1 -- 172.21.15.7:0/2022707853 wait complete. 2024-03-23T13:19:24.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:23 smithi130 ceph-mon[36225]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T13:19:24.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:23 smithi155 ceph-mon[36157]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T13:19:24.755 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T13:19:24.763 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-23T13:19:24.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-23T13:19:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:19:26.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:25 smithi007 ceph-mon[32184]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T13:19:26.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:25 smithi007 ceph-mon[32184]: from='client.25024 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:26.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:25 smithi130 ceph-mon[36225]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T13:19:26.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:25 smithi130 ceph-mon[36225]: from='client.25024 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:26.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:25 smithi155 ceph-mon[36157]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T13:19:26.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:25 smithi155 ceph-mon[36157]: from='client.25024 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T13:19:26.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 -- 172.21.15.7:0/1684124881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4105640 msgr2=0x7f94b4107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:26.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 --2- 172.21.15.7:0/1684124881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4105640 0x7f94b4107a30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f949c009990 tx=0x7f949c009ca0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 -- 172.21.15.7:0/1684124881 shutdown_connections 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 --2- 172.21.15.7:0/1684124881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b4107f70 0x7f94b410a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 --2- 172.21.15.7:0/1684124881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4102d10 0x7f94b4105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 --2- 172.21.15.7:0/1684124881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4105640 0x7f94b4107a30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 -- 172.21.15.7:0/1684124881 >> 172.21.15.7:0/1684124881 conn(0x7f94b40fc6c0 msgr2=0x7f94b40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 -- 172.21.15.7:0/1684124881 shutdown_connections 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 -- 172.21.15.7:0/1684124881 wait complete. 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 Processor -- start 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.676+0000 7f94b8e0b700 1 -- start start 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b8e0b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b4102d10 0x7f94b419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b8e0b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4105640 0x7f94b419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b8e0b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4107f70 0x7f94b41a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b8e0b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94b41a1f50 con 0x7f94b4105640 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b8e0b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f94b41a2090 con 0x7f94b4107f70 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b8e0b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94b41a2220 con 0x7f94b4102d10 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4105640 0x7f94b419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b4102d10 0x7f94b419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4105640 0x7f94b419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60708/0 (socket says 172.21.15.7:60708) 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b1d9b700 1 -- 172.21.15.7:0/3846239991 learned_addr learned my addr 172.21.15.7:0/3846239991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b2d9d700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4107f70 0x7f94b41a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b259c700 1 -- 172.21.15.7:0/3846239991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4107f70 msgr2=0x7f94b41a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b259c700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4107f70 0x7f94b41a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b259c700 1 -- 172.21.15.7:0/3846239991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4105640 msgr2=0x7f94b419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b259c700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4105640 0x7f94b419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b259c700 1 -- 172.21.15.7:0/3846239991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949c0095f0 con 0x7f94b4102d10 2024-03-23T13:19:26.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b1d9b700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4105640 0x7f94b419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:26.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.677+0000 7f94b259c700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b4102d10 0x7f94b419a3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f94a400ef40 tx=0x7f94a400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:26.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.678+0000 7f94ab7fe700 1 -- 172.21.15.7:0/3846239991 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a400c9c0 con 0x7f94b4102d10 2024-03-23T13:19:26.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.678+0000 7f94ab7fe700 1 -- 172.21.15.7:0/3846239991 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a400f040 con 0x7f94b4102d10 2024-03-23T13:19:26.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.678+0000 7f94b8e0b700 1 -- 172.21.15.7:0/3846239991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b41a24a0 con 0x7f94b4102d10 2024-03-23T13:19:26.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.678+0000 7f94ab7fe700 1 -- 172.21.15.7:0/3846239991 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a40146e0 con 0x7f94b4102d10 2024-03-23T13:19:26.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.678+0000 7f94b8e0b700 1 -- 172.21.15.7:0/3846239991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b41a29f0 con 0x7f94b4102d10 2024-03-23T13:19:26.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.678+0000 7f94b8e0b700 1 -- 172.21.15.7:0/3846239991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b4109af0 con 0x7f94b4102d10 2024-03-23T13:19:26.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.680+0000 7f94ab7fe700 1 -- 172.21.15.7:0/3846239991 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f94a400cb20 con 0x7f94b4102d10 2024-03-23T13:19:26.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.680+0000 7f94ab7fe700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f94a00a21d0 0x7f94a00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:26.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.680+0000 7f94b1d9b700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f94a00a21d0 0x7f94a00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:26.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.680+0000 7f94ab7fe700 1 -- 172.21.15.7:0/3846239991 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f94a40cde80 con 0x7f94b4102d10 2024-03-23T13:19:26.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.681+0000 7f94b1d9b700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f94a00a21d0 0x7f94a00a4690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f949c00b2e0 tx=0x7f949c00b7c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.684+0000 7f94ab7fe700 1 -- 172.21.15.7:0/3846239991 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f94a4096e60 con 0x7f94b4102d10 2024-03-23T13:19:26.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.933+0000 7f94b8e0b700 1 -- 172.21.15.7:0/3846239991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f94b410dba0 con 0x7f94b4102d10 2024-03-23T13:19:26.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.934+0000 7f94ab7fe700 1 -- 172.21.15.7:0/3846239991 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3080 (secure 0 0 0) 0x7f94a409a520 con 0x7f94b4102d10 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:e26 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 26 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-23T13:17:40.597309+0000 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-23T13:17:48.259350+0000 2024-03-23T13:19:26.935 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 300 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 5 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1,2,3,4 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=24503,1=24496,2=24475,3=24470,4=24463} 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-23T13:19:26.936 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 0 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 24503 members: 24470,24463,24475,24503,24496 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b{0:24503} state up:active seq 3 addr [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e{0:24440} state up:standby-replay seq 1 addr [v2:172.21.15.130:6834/2599360099,v1:172.21.15.130:6835/2599360099] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.i{1:24496} state up:active seq 6 addr [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.l{1:24415} state up:standby-replay seq 1 addr [v2:172.21.15.155:6832/1319723529,v1:172.21.15.155:6833/1319723529] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c{2:24475} state up:active seq 7 addr [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.k{2:24413} state up:standby-replay seq 1 addr [v2:172.21.15.130:6832/3818242586,v1:172.21.15.130:6833/3818242586] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.h{3:24470} state up:active seq 7 addr [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a{3:14649} state up:standby-replay seq 1 addr [v2:172.21.15.7:6840/3415570481,v1:172.21.15.7:6841/3415570481] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f{4:24463} state up:active seq 8 addr [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.g{4:14637} state up:standby-replay seq 1 addr [v2:172.21.15.7:6838/388542752,v1:172.21.15.7:6839/388542752] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:26.938 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-23T13:19:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:26.938 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.j{-1:14559} state up:standby seq 1 addr [v2:172.21.15.7:6834/3229137818,v1:172.21.15.7:6835/3229137818] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.938 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d{-1:14571} state up:standby seq 1 addr [v2:172.21.15.7:6836/746475377,v1:172.21.15.7:6837/746475377] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 -- 172.21.15.7:0/3846239991 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f94a00a21d0 msgr2=0x7f94a00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f94a00a21d0 0x7f94a00a4690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f949c00b2e0 tx=0x7f949c00b7c0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 -- 172.21.15.7:0/3846239991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b4102d10 msgr2=0x7f94b419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b4102d10 0x7f94b419a3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f94a400ef40 tx=0x7f94a400c3a0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 -- 172.21.15.7:0/3846239991 shutdown_connections 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f94a00a21d0 0x7f94a00a4690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4107f70 0x7f94b41a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b4102d10 0x7f94b419a3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 --2- 172.21.15.7:0/3846239991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4105640 0x7f94b419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 -- 172.21.15.7:0/3846239991 >> 172.21.15.7:0/3846239991 conn(0x7f94b40fc6c0 msgr2=0x7f94b4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 -- 172.21.15.7:0/3846239991 shutdown_connections 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:26.937+0000 7f94b8e0b700 1 -- 172.21.15.7:0/3846239991 wait complete. 2024-03-23T13:19:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-23T13:19:27.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-23T13:19:27.824 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:19:28.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:27 smithi007 ceph-mon[32184]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T13:19:28.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:27 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3846239991' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T13:19:28.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:27 smithi130 ceph-mon[36225]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T13:19:28.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:27 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3846239991' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T13:19:28.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:27 smithi155 ceph-mon[36157]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T13:19:28.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:27 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3846239991' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T13:19:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.581+0000 7fb889217700 1 -- 172.21.15.7:0/2291927404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb884102230 msgr2=0x7fb884102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.581+0000 7fb889217700 1 --2- 172.21.15.7:0/2291927404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb884102230 0x7fb884102650 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb8740099c0 tx=0x7fb874009cd0 comp rx=0 tx=0).stop 2024-03-23T13:19:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.581+0000 7fb889217700 1 -- 172.21.15.7:0/2291927404 shutdown_connections 2024-03-23T13:19:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.581+0000 7fb889217700 1 --2- 172.21.15.7:0/2291927404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb884103dd0 0x7fb88410a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.581+0000 7fb889217700 1 --2- 172.21.15.7:0/2291927404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb884102230 0x7fb884102650 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.581+0000 7fb889217700 1 --2- 172.21.15.7:0/2291927404 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb884103430 0x7fb884103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.581+0000 7fb889217700 1 -- 172.21.15.7:0/2291927404 >> 172.21.15.7:0/2291927404 conn(0x7fb8840fd810 msgr2=0x7fb8840ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.581+0000 7fb889217700 1 -- 172.21.15.7:0/2291927404 shutdown_connections 2024-03-23T13:19:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.581+0000 7fb889217700 1 -- 172.21.15.7:0/2291927404 wait complete. 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.582+0000 7fb889217700 1 Processor -- start 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.582+0000 7fb889217700 1 -- start start 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb889217700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb884102230 0x7fb8841981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb889217700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb884103430 0x7fb884198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb889217700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb884103dd0 0x7fb88419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb889217700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb88419fd80 con 0x7fb884102230 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb889217700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb88419fec0 con 0x7fb884103dd0 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb889217700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841a0050 con 0x7fb884103430 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb882d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb884102230 0x7fb8841981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb88259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb884103430 0x7fb884198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb882d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb884102230 0x7fb8841981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60724/0 (socket says 172.21.15.7:60724) 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb882d9d700 1 -- 172.21.15.7:0/2224439679 learned_addr learned my addr 172.21.15.7:0/2224439679 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb88359e700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb884103dd0 0x7fb88419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb882d9d700 1 -- 172.21.15.7:0/2224439679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb884103430 msgr2=0x7fb884198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb882d9d700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb884103430 0x7fb884198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb882d9d700 1 -- 172.21.15.7:0/2224439679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb884103dd0 msgr2=0x7fb88419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb882d9d700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb884103dd0 0x7fb88419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb882d9d700 1 -- 172.21.15.7:0/2224439679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8740095f0 con 0x7fb884102230 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb88359e700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb884103dd0 0x7fb88419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb88259c700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb884103430 0x7fb884198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.583+0000 7fb882d9d700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb884102230 0x7fb8841981f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb874009fd0 tx=0x7fb87400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.584+0000 7fb86bfff700 1 -- 172.21.15.7:0/2224439679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87401d070 con 0x7fb884102230 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.584+0000 7fb889217700 1 -- 172.21.15.7:0/2224439679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8841a02d0 con 0x7fb884102230 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.584+0000 7fb86bfff700 1 -- 172.21.15.7:0/2224439679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb874005940 con 0x7fb884102230 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.584+0000 7fb889217700 1 -- 172.21.15.7:0/2224439679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8841a07c0 con 0x7fb884102230 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.584+0000 7fb86bfff700 1 -- 172.21.15.7:0/2224439679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874022e90 con 0x7fb884102230 2024-03-23T13:19:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.585+0000 7fb889217700 1 -- 172.21.15.7:0/2224439679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb870005360 con 0x7fb884102230 2024-03-23T13:19:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.586+0000 7fb86bfff700 1 -- 172.21.15.7:0/2224439679 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fb874022470 con 0x7fb884102230 2024-03-23T13:19:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.587+0000 7fb86bfff700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb86c0a2150 0x7fb86c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.587+0000 7fb88259c700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb86c0a2150 0x7fb86c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.587+0000 7fb86bfff700 1 -- 172.21.15.7:0/2224439679 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb8740cee00 con 0x7fb884102230 2024-03-23T13:19:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.587+0000 7fb88259c700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb86c0a2150 0x7fb86c0a4610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb878011490 tx=0x7fb878009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.591+0000 7fb86bfff700 1 -- 172.21.15.7:0/2224439679 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb874097f30 con 0x7fb884102230 2024-03-23T13:19:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.794+0000 7fb889217700 1 -- 172.21.15.7:0/2224439679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb8700058e0 con 0x7fb884102230 2024-03-23T13:19:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.795+0000 7fb86bfff700 1 -- 172.21.15.7:0/2224439679 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5245 (secure 0 0 0) 0x7fb87409b5f0 con 0x7fb884102230 2024-03-23T13:19:29.796 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 87 2024-03-23T13:19:29.796 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-23T13:09:39.213712+0000 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-23T13:18:43.667397+0000 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 26 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client luminous 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-23T13:19:29.797 INFO:teuthology.orchestra.run.smithi007.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 12 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6802/2113068565,v1:172.21.15.7:6803/2113068565] [v2:172.21.15.7:6804/2113068565,v1:172.21.15.7:6805/2113068565] exists,up 0da10b6e-bba6-44c4-ae11-5378f134eb22 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6800/704785998,v1:172.21.15.130:6801/704785998] [v2:172.21.15.130:6802/704785998,v1:172.21.15.130:6803/704785998] exists,up 52bc95ae-72ec-43dd-b7b7-7a88b6593c0f 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/2256065708,v1:172.21.15.155:6801/2256065708] [v2:172.21.15.155:6802/2256065708,v1:172.21.15.155:6803/2256065708] exists,up 2013fd43-79f2-420b-8518-046fce065cd7 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6810/28573179,v1:172.21.15.7:6811/28573179] [v2:172.21.15.7:6812/28573179,v1:172.21.15.7:6813/28573179] exists,up 1a418507-c9a0-4945-aca6-43c156f4f96b 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6808/568360270,v1:172.21.15.130:6809/568360270] [v2:172.21.15.130:6810/568360270,v1:172.21.15.130:6811/568360270] exists,up 258432a8-3995-4878-928f-1bc7cc486699 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6808/3878988779,v1:172.21.15.155:6809/3878988779] [v2:172.21.15.155:6810/3878988779,v1:172.21.15.155:6811/3878988779] exists,up f115a6d4-3043-48d6-a29e-b367c62dd418 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6818/2961397524,v1:172.21.15.7:6819/2961397524] [v2:172.21.15.7:6820/2961397524,v1:172.21.15.7:6821/2961397524] exists,up 16353022-32db-487a-99e8-d8fe28fdaef0 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 51 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6816/869404947,v1:172.21.15.130:6817/869404947] [v2:172.21.15.130:6818/869404947,v1:172.21.15.130:6819/869404947] exists,up c3b2652e-e404-4893-aff0-5049956327ad 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/2565353660,v1:172.21.15.155:6817/2565353660] [v2:172.21.15.155:6818/2565353660,v1:172.21.15.155:6819/2565353660] exists,up 46014832-6a35-46b2-a918-ef9179dc3bf1 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6826/439190456,v1:172.21.15.7:6827/439190456] [v2:172.21.15.7:6828/439190456,v1:172.21.15.7:6829/439190456] exists,up 6fdeaf21-f0a5-4dff-ac58-8bb059972ffe 2024-03-23T13:19:29.798 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6824/1468057097,v1:172.21.15.130:6825/1468057097] [v2:172.21.15.130:6826/1468057097,v1:172.21.15.130:6827/1468057097] exists,up 06ebd1ee-dd0b-424f-9e4d-113d31950c5f 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/2887615966,v1:172.21.15.155:6825/2887615966] [v2:172.21.15.155:6826/2887615966,v1:172.21.15.155:6827/2887615966] exists,up fc34a0d3-024e-4b1f-907b-147a3fd1298d 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.5 [8,11] 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.8 [2,11] 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.1d [2,4] 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.25 [5,9] 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.31 [4,6] 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.36 [4,9] 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.38 [3,9] 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.3a [4,8] 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/2507019439 expires 2024-03-24T13:10:41.843648+0000 2024-03-23T13:19:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1286669735 expires 2024-03-24T13:10:41.843648+0000 2024-03-23T13:19:29.800 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3684792738 expires 2024-03-24T13:10:41.843648+0000 2024-03-23T13:19:29.800 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/3945637763 expires 2024-03-24T13:10:41.843648+0000 2024-03-23T13:19:29.800 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/742242630 expires 2024-03-24T13:10:15.062657+0000 2024-03-23T13:19:29.800 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/310337452 expires 2024-03-24T13:10:15.062657+0000 2024-03-23T13:19:29.800 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/310337452 expires 2024-03-24T13:10:15.062657+0000 2024-03-23T13:19:29.800 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1496728145 expires 2024-03-24T13:10:15.062657+0000 2024-03-23T13:19:29.800 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/3945637763 expires 2024-03-24T13:10:41.843648+0000 2024-03-23T13:19:29.800 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1695133666 expires 2024-03-24T13:10:15.062657+0000 2024-03-23T13:19:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.798+0000 7fb889217700 1 -- 172.21.15.7:0/2224439679 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb86c0a2150 msgr2=0x7fb86c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.798+0000 7fb889217700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb86c0a2150 0x7fb86c0a4610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb878011490 tx=0x7fb878009260 comp rx=0 tx=0).stop 2024-03-23T13:19:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.798+0000 7fb889217700 1 -- 172.21.15.7:0/2224439679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb884102230 msgr2=0x7fb8841981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.798+0000 7fb889217700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb884102230 0x7fb8841981f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb874009fd0 tx=0x7fb87400f690 comp rx=0 tx=0).stop 2024-03-23T13:19:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.799+0000 7fb889217700 1 -- 172.21.15.7:0/2224439679 shutdown_connections 2024-03-23T13:19:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.799+0000 7fb889217700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb86c0a2150 0x7fb86c0a4610 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.799+0000 7fb889217700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb884103dd0 0x7fb88419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.799+0000 7fb889217700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb884102230 0x7fb8841981f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.799+0000 7fb889217700 1 --2- 172.21.15.7:0/2224439679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb884103430 0x7fb884198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.799+0000 7fb889217700 1 -- 172.21.15.7:0/2224439679 >> 172.21.15.7:0/2224439679 conn(0x7fb8840fd810 msgr2=0x7fb8841067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.799+0000 7fb889217700 1 -- 172.21.15.7:0/2224439679 shutdown_connections 2024-03-23T13:19:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:29.799+0000 7fb889217700 1 -- 172.21.15.7:0/2224439679 wait complete. 2024-03-23T13:19:30.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:29 smithi007 ceph-mon[32184]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-23T13:19:30.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:29 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2224439679' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T13:19:30.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:29 smithi130 ceph-mon[36225]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-23T13:19:30.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:29 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2224439679' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T13:19:30.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:29 smithi155 ceph-mon[36157]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-23T13:19:30.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:29 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2224439679' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T13:19:30.507 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-23T13:19:30.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:19:30.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.628+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/507928810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d48100fb0 msgr2=0x7f7d48101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:30.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.628+0000 7f7d4e3f4640 1 --2- 172.21.15.7:0/507928810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d48100fb0 0x7f7d48101430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800aa10 tx=0x7f7d3802cf70 comp rx=0 tx=0).stop 2024-03-23T13:19:30.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.628+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/507928810 shutdown_connections 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.628+0000 7f7d4e3f4640 1 --2- 172.21.15.7:0/507928810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d48101970 0x7f7d48108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.628+0000 7f7d4e3f4640 1 --2- 172.21.15.7:0/507928810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d48100fb0 0x7f7d48101430 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.628+0000 7f7d4e3f4640 1 --2- 172.21.15.7:0/507928810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d480ffdc0 0x7f7d481001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.628+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/507928810 >> 172.21.15.7:0/507928810 conn(0x7f7d480fb330 msgr2=0x7f7d480fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.629+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/507928810 shutdown_connections 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.629+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/507928810 wait complete. 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.629+0000 7f7d4e3f4640 1 Processor -- start 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.629+0000 7f7d4e3f4640 1 -- start start 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.629+0000 7f7d4e3f4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d480ffdc0 0x7f7d48197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.629+0000 7f7d4e3f4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d48100fb0 0x7f7d48197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.629+0000 7f7d4e3f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d48101970 0x7f7d4819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.629+0000 7f7d4e3f4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4819f400 con 0x7f7d48101970 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.629+0000 7f7d4e3f4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4819f570 con 0x7f7d480ffdc0 2024-03-23T13:19:30.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.629+0000 7f7d4e3f4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4819f730 con 0x7f7d48100fb0 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d4c991640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d48101970 0x7f7d4819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d4c991640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d48101970 0x7f7d4819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60746/0 (socket says 172.21.15.7:60746) 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d47fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d480ffdc0 0x7f7d48197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d4c991640 1 -- 172.21.15.7:0/1713928145 learned_addr learned my addr 172.21.15.7:0/1713928145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d477fe640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d48100fb0 0x7f7d48197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d47fff640 1 -- 172.21.15.7:0/1713928145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d48100fb0 msgr2=0x7f7d48197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d47fff640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d48100fb0 0x7f7d48197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d47fff640 1 -- 172.21.15.7:0/1713928145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d48101970 msgr2=0x7f7d4819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d47fff640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d48101970 0x7f7d4819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d47fff640 1 -- 172.21.15.7:0/1713928145 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3800a670 con 0x7f7d480ffdc0 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d4c991640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d48101970 0x7f7d4819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d47fff640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d480ffdc0 0x7f7d48197910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00e9c0 tx=0x7f7d3c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d457fa640 1 -- 172.21.15.7:0/1713928145 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c00ccc0 con 0x7f7d480ffdc0 2024-03-23T13:19:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d457fa640 1 -- 172.21.15.7:0/1713928145 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d3c00ce20 con 0x7f7d480ffdc0 2024-03-23T13:19:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/1713928145 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4819fa10 con 0x7f7d480ffdc0 2024-03-23T13:19:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d457fa640 1 -- 172.21.15.7:0/1713928145 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c010430 con 0x7f7d480ffdc0 2024-03-23T13:19:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.630+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/1713928145 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4819fea0 con 0x7f7d480ffdc0 2024-03-23T13:19:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.631+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/1713928145 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d481070e0 con 0x7f7d480ffdc0 2024-03-23T13:19:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.632+0000 7f7d457fa640 1 -- 172.21.15.7:0/1713928145 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f7d3c004000 con 0x7f7d480ffdc0 2024-03-23T13:19:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.633+0000 7f7d457fa640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7d300a20c0 0x7f7d300a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.633+0000 7f7d477fe640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7d300a20c0 0x7f7d300a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.633+0000 7f7d457fa640 1 -- 172.21.15.7:0/1713928145 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7d3c014070 con 0x7f7d480ffdc0 2024-03-23T13:19:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.633+0000 7f7d477fe640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7d300a20c0 0x7f7d300a4580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d38004c90 tx=0x7f7d38005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.636+0000 7f7d457fa640 1 -- 172.21.15.7:0/1713928145 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7d3c096f90 con 0x7f7d480ffdc0 2024-03-23T13:19:30.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.820+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/1713928145 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d4810b140 con 0x7f7d480ffdc0 2024-03-23T13:19:30.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.822+0000 7f7d457fa640 1 -- 172.21.15.7:0/1713928145 <== mon.2 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22558 (secure 0 0 0) 0x7f7d3c09a650 con 0x7f7d480ffdc0 2024-03-23T13:19:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:30.823 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:18:43.667397+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.823+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/1713928145 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7d300a20c0 msgr2=0x7f7d300a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.823+0000 7f7d4e3f4640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7d300a20c0 0x7f7d300a4580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d38004c90 tx=0x7f7d38005fd0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.823+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/1713928145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d480ffdc0 msgr2=0x7f7d48197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.824+0000 7f7d4e3f4640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d480ffdc0 0x7f7d48197910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00e9c0 tx=0x7f7d3c00ee80 comp rx=0 tx=0).stop 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.824+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/1713928145 shutdown_connections 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.824+0000 7f7d4e3f4640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7d300a20c0 0x7f7d300a4580 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.824+0000 7f7d4e3f4640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d48101970 0x7f7d4819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.824+0000 7f7d4e3f4640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d48100fb0 0x7f7d48197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.824+0000 7f7d4e3f4640 1 --2- 172.21.15.7:0/1713928145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d480ffdc0 0x7f7d48197910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.824+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/1713928145 >> 172.21.15.7:0/1713928145 conn(0x7f7d480fb330 msgr2=0x7f7d481042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.824+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/1713928145 shutdown_connections 2024-03-23T13:19:30.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.824+0000 7f7d4e3f4640 1 -- 172.21.15.7:0/1713928145 wait complete. 2024-03-23T13:19:30.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T13:13:08.186719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T13:17:38.843887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T13:17:39.788489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T13:19:30.836 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T13:19:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.941+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/487001332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a54101800 msgr2=0x7f8a54101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.941+0000 7f8a5c2e8640 1 --2- 172.21.15.7:0/487001332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a54101800 0x7f8a54101c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009860 tx=0x7f8a4402bb70 comp rx=0 tx=0).stop 2024-03-23T13:19:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.941+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/487001332 shutdown_connections 2024-03-23T13:19:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.941+0000 7f8a5c2e8640 1 --2- 172.21.15.7:0/487001332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a541021c0 0x7f8a54108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.941+0000 7f8a5c2e8640 1 --2- 172.21.15.7:0/487001332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a54101800 0x7f8a54101c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.941+0000 7f8a5c2e8640 1 --2- 172.21.15.7:0/487001332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a54100610 0x7f8a54100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.941+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/487001332 >> 172.21.15.7:0/487001332 conn(0x7f8a540fbba0 msgr2=0x7f8a540fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.941+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/487001332 shutdown_connections 2024-03-23T13:19:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.941+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/487001332 wait complete. 2024-03-23T13:19:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a5c2e8640 1 Processor -- start 2024-03-23T13:19:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a5c2e8640 1 -- start start 2024-03-23T13:19:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a5c2e8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a54100610 0x7f8a54196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a5c2e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a54101800 0x7f8a54196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a5c2e8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a541021c0 0x7f8a5419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a5c2e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5419e140 con 0x7f8a54101800 2024-03-23T13:19:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a5c2e8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5419e2b0 con 0x7f8a541021c0 2024-03-23T13:19:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a5c2e8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5419e470 con 0x7f8a54100610 2024-03-23T13:19:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a59883640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a54101800 0x7f8a54196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a59883640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a54101800 0x7f8a54196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60772/0 (socket says 172.21.15.7:60772) 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a59883640 1 -- 172.21.15.7:0/1732901879 learned_addr learned my addr 172.21.15.7:0/1732901879 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a5a084640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a54100610 0x7f8a54196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a5a885640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a541021c0 0x7f8a5419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a59883640 1 -- 172.21.15.7:0/1732901879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a54100610 msgr2=0x7f8a54196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a59883640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a54100610 0x7f8a54196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a59883640 1 -- 172.21.15.7:0/1732901879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a541021c0 msgr2=0x7f8a5419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a59883640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a541021c0 0x7f8a5419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.942+0000 7f8a59883640 1 -- 172.21.15.7:0/1732901879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c009900 con 0x7f8a54101800 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.943+0000 7f8a59883640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a54101800 0x7f8a54196b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8a44034b10 tx=0x7f8a440045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.943+0000 7f8a4b7fe640 1 -- 172.21.15.7:0/1732901879 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44005020 con 0x7f8a54101800 2024-03-23T13:19:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.943+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/1732901879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a44009490 con 0x7f8a54101800 2024-03-23T13:19:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.943+0000 7f8a4b7fe640 1 -- 172.21.15.7:0/1732901879 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4402c510 con 0x7f8a54101800 2024-03-23T13:19:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.943+0000 7f8a4b7fe640 1 -- 172.21.15.7:0/1732901879 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44004a60 con 0x7f8a54101800 2024-03-23T13:19:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.943+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/1732901879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5419e920 con 0x7f8a54101800 2024-03-23T13:19:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.944+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/1732901879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a54107930 con 0x7f8a54101800 2024-03-23T13:19:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.945+0000 7f8a4b7fe640 1 -- 172.21.15.7:0/1732901879 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f8a4403b040 con 0x7f8a54101800 2024-03-23T13:19:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.945+0000 7f8a4b7fe640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8a400a20c0 0x7f8a400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.945+0000 7f8a5a084640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8a400a20c0 0x7f8a400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.945+0000 7f8a4b7fe640 1 -- 172.21.15.7:0/1732901879 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8a440ef600 con 0x7f8a54101800 2024-03-23T13:19:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.946+0000 7f8a5a084640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8a400a20c0 0x7f8a400a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8a54101660 tx=0x7f8a50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:30.950+0000 7f8a4b7fe640 1 -- 172.21.15.7:0/1732901879 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f8a440b8630 con 0x7f8a54101800 2024-03-23T13:19:31.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.132+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/1732901879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a5402d160 con 0x7f8a54101800 2024-03-23T13:19:31.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.133+0000 7f8a4b7fe640 1 -- 172.21.15.7:0/1732901879 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f8a440bbcf0 con 0x7f8a54101800 2024-03-23T13:19:31.134 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-23T13:19:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/1732901879 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8a400a20c0 msgr2=0x7f8a400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8a400a20c0 0x7f8a400a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8a54101660 tx=0x7f8a50005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/1732901879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a54101800 msgr2=0x7f8a54196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a54101800 0x7f8a54196b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8a44034b10 tx=0x7f8a440045c0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/1732901879 shutdown_connections 2024-03-23T13:19:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8a400a20c0 0x7f8a400a4580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a541021c0 0x7f8a5419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a54101800 0x7f8a54196b90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 --2- 172.21.15.7:0/1732901879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a54100610 0x7f8a54196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/1732901879 >> 172.21.15.7:0/1732901879 conn(0x7f8a540fbba0 msgr2=0x7f8a54104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/1732901879 shutdown_connections 2024-03-23T13:19:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.135+0000 7f8a5c2e8640 1 -- 172.21.15.7:0/1732901879 wait complete. 2024-03-23T13:19:31.147 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T13:19:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.251+0000 7f4544380640 1 -- 172.21.15.7:0/597516930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453c101820 msgr2=0x7f453c101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.251+0000 7f4544380640 1 --2- 172.21.15.7:0/597516930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453c101820 0x7f453c101ca0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4530009900 tx=0x7f453002bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.251+0000 7f4544380640 1 -- 172.21.15.7:0/597516930 shutdown_connections 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.251+0000 7f4544380640 1 --2- 172.21.15.7:0/597516930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f453c1021e0 0x7f453c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.251+0000 7f4544380640 1 --2- 172.21.15.7:0/597516930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453c101820 0x7f453c101ca0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.251+0000 7f4544380640 1 --2- 172.21.15.7:0/597516930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f453c100630 0x7f453c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 -- 172.21.15.7:0/597516930 >> 172.21.15.7:0/597516930 conn(0x7f453c0fbbc0 msgr2=0x7f453c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 -- 172.21.15.7:0/597516930 shutdown_connections 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 -- 172.21.15.7:0/597516930 wait complete. 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 Processor -- start 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 -- start start 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f453c100630 0x7f453c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f453c101820 0x7f453c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453c1021e0 0x7f453c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f453c19f320 con 0x7f453c1021e0 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f453c19f490 con 0x7f453c101820 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f4544380640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f453c19f650 con 0x7f453c100630 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f454291d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453c1021e0 0x7f453c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f454291d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453c1021e0 0x7f453c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60794/0 (socket says 172.21.15.7:60794) 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f454291d640 1 -- 172.21.15.7:0/2387125517 learned_addr learned my addr 172.21.15.7:0/2387125517 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f454211c640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f453c100630 0x7f453c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.252+0000 7f454191b640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f453c101820 0x7f453c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f454291d640 1 -- 172.21.15.7:0/2387125517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f453c100630 msgr2=0x7f453c06a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f454291d640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f453c100630 0x7f453c06a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f454291d640 1 -- 172.21.15.7:0/2387125517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f453c101820 msgr2=0x7f453c19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f454291d640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f453c101820 0x7f453c19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f454291d640 1 -- 172.21.15.7:0/2387125517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4530009560 con 0x7f453c1021e0 2024-03-23T13:19:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f454291d640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453c1021e0 0x7f453c19eb40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f452c00eb10 tx=0x7f452c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f453b7fe640 1 -- 172.21.15.7:0/2387125517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c004b20 con 0x7f453c1021e0 2024-03-23T13:19:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f453b7fe640 1 -- 172.21.15.7:0/2387125517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f452c004c80 con 0x7f453c1021e0 2024-03-23T13:19:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f4544380640 1 -- 172.21.15.7:0/2387125517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453c19f8d0 con 0x7f453c1021e0 2024-03-23T13:19:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f453b7fe640 1 -- 172.21.15.7:0/2387125517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c010650 con 0x7f453c1021e0 2024-03-23T13:19:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.253+0000 7f4544380640 1 -- 172.21.15.7:0/2387125517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453c19fdb0 con 0x7f453c1021e0 2024-03-23T13:19:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.254+0000 7f4544380640 1 -- 172.21.15.7:0/2387125517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453c1079d0 con 0x7f453c1021e0 2024-03-23T13:19:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.255+0000 7f453b7fe640 1 -- 172.21.15.7:0/2387125517 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f452c005020 con 0x7f453c1021e0 2024-03-23T13:19:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.256+0000 7f453b7fe640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f45240a2140 0x7f45240a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.256+0000 7f454211c640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f45240a2140 0x7f45240a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.256+0000 7f453b7fe640 1 -- 172.21.15.7:0/2387125517 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f452c014070 con 0x7f453c1021e0 2024-03-23T13:19:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.256+0000 7f454211c640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f45240a2140 0x7f45240a4600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f453c101680 tx=0x7f4534005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.260+0000 7f453b7fe640 1 -- 172.21.15.7:0/2387125517 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f452c0971b0 con 0x7f453c1021e0 2024-03-23T13:19:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.442+0000 7f4544380640 1 -- 172.21.15.7:0/2387125517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f453c02d230 con 0x7f453c1021e0 2024-03-23T13:19:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.442+0000 7f453b7fe640 1 -- 172.21.15.7:0/2387125517 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f452c09a870 con 0x7f453c1021e0 2024-03-23T13:19:31.443 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 -- 172.21.15.7:0/2387125517 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f45240a2140 msgr2=0x7f45240a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f45240a2140 0x7f45240a4600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f453c101680 tx=0x7f4534005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 -- 172.21.15.7:0/2387125517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453c1021e0 msgr2=0x7f453c19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453c1021e0 0x7f453c19eb40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f452c00eb10 tx=0x7f452c00c2d0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 -- 172.21.15.7:0/2387125517 shutdown_connections 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f45240a2140 0x7f45240a4600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453c1021e0 0x7f453c19eb40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f453c101820 0x7f453c19c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 --2- 172.21.15.7:0/2387125517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f453c100630 0x7f453c06a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 -- 172.21.15.7:0/2387125517 >> 172.21.15.7:0/2387125517 conn(0x7f453c0fbbc0 msgr2=0x7f453c104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 -- 172.21.15.7:0/2387125517 shutdown_connections 2024-03-23T13:19:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.445+0000 7f4544380640 1 -- 172.21.15.7:0/2387125517 wait complete. 2024-03-23T13:19:31.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T13:19:31.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.561+0000 7f7814156640 1 -- 172.21.15.7:0/3833491373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f780c102910 msgr2=0x7f780c104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.561+0000 7f7814156640 1 --2- 172.21.15.7:0/3833491373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f780c102910 0x7f780c104d80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009900 tx=0x7f77fc02bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:31.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.561+0000 7f7814156640 1 -- 172.21.15.7:0/3833491373 shutdown_connections 2024-03-23T13:19:31.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.561+0000 7f7814156640 1 --2- 172.21.15.7:0/3833491373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f780c1052c0 0x7f780c105740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.561+0000 7f7814156640 1 --2- 172.21.15.7:0/3833491373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f780c102910 0x7f780c104d80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.561+0000 7f7814156640 1 --2- 172.21.15.7:0/3833491373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f780c0fffa0 0x7f780c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.561+0000 7f7814156640 1 -- 172.21.15.7:0/3833491373 >> 172.21.15.7:0/3833491373 conn(0x7f780c0fbbe0 msgr2=0x7f780c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.561+0000 7f7814156640 1 -- 172.21.15.7:0/3833491373 shutdown_connections 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.561+0000 7f7814156640 1 -- 172.21.15.7:0/3833491373 wait complete. 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f7814156640 1 Processor -- start 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f7814156640 1 -- start start 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f7814156640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f780c0fffa0 0x7f780c19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f7814156640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f780c102910 0x7f780c19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f7814156640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f780c1052c0 0x7f780c1a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f7814156640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1a5860 con 0x7f780c1052c0 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f7814156640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1a59d0 con 0x7f780c102910 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f7814156640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1a5b90 con 0x7f780c0fffa0 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78126f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f780c1052c0 0x7f780c1a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78126f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f780c1052c0 0x7f780c1a5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60804/0 (socket says 172.21.15.7:60804) 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f7811ef2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f780c0fffa0 0x7f780c19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78126f3640 1 -- 172.21.15.7:0/1489205756 learned_addr learned my addr 172.21.15.7:0/1489205756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78116f1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f780c102910 0x7f780c19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78126f3640 1 -- 172.21.15.7:0/1489205756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f780c0fffa0 msgr2=0x7f780c19dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78126f3640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f780c0fffa0 0x7f780c19dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78126f3640 1 -- 172.21.15.7:0/1489205756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f780c102910 msgr2=0x7f780c19e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78126f3640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f780c102910 0x7f780c19e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78126f3640 1 -- 172.21.15.7:0/1489205756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc009560 con 0x7f780c1052c0 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78116f1640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f780c102910 0x7f780c19e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f78126f3640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f780c1052c0 0x7f780c1a5320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f780400eb10 tx=0x7f780400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.562+0000 7f7802ffd640 1 -- 172.21.15.7:0/1489205756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804004b20 con 0x7f780c1052c0 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.563+0000 7f7814156640 1 -- 172.21.15.7:0/1489205756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780c1a5e70 con 0x7f780c1052c0 2024-03-23T13:19:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.563+0000 7f7802ffd640 1 -- 172.21.15.7:0/1489205756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7804004c80 con 0x7f780c1052c0 2024-03-23T13:19:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.563+0000 7f7802ffd640 1 -- 172.21.15.7:0/1489205756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804010650 con 0x7f780c1052c0 2024-03-23T13:19:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.563+0000 7f7814156640 1 -- 172.21.15.7:0/1489205756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780c1a62f0 con 0x7f780c1052c0 2024-03-23T13:19:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.564+0000 7f7814156640 1 -- 172.21.15.7:0/1489205756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780c0fd6d0 con 0x7f780c1052c0 2024-03-23T13:19:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.565+0000 7f7802ffd640 1 -- 172.21.15.7:0/1489205756 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f7804010870 con 0x7f780c1052c0 2024-03-23T13:19:31.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.566+0000 7f7802ffd640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77f80a20c0 0x7f77f80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.566+0000 7f7802ffd640 1 -- 172.21.15.7:0/1489205756 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7804014070 con 0x7f780c1052c0 2024-03-23T13:19:31.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.566+0000 7f7811ef2640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77f80a20c0 0x7f77f80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.566+0000 7f7811ef2640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77f80a20c0 0x7f77f80a4580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7808009bd0 tx=0x7f7808005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.570+0000 7f7802ffd640 1 -- 172.21.15.7:0/1489205756 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f78040973b0 con 0x7f780c1052c0 2024-03-23T13:19:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.754+0000 7f7814156640 1 -- 172.21.15.7:0/1489205756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f780c1138c0 con 0x7f780c1052c0 2024-03-23T13:19:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.755+0000 7f7802ffd640 1 -- 172.21.15.7:0/1489205756 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f780409aa70 con 0x7f780c1052c0 2024-03-23T13:19:31.756 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-23T13:19:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 -- 172.21.15.7:0/1489205756 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77f80a20c0 msgr2=0x7f77f80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77f80a20c0 0x7f77f80a4580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7808009bd0 tx=0x7f7808005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 -- 172.21.15.7:0/1489205756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f780c1052c0 msgr2=0x7f780c1a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f780c1052c0 0x7f780c1a5320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f780400eb10 tx=0x7f780400c2d0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 -- 172.21.15.7:0/1489205756 shutdown_connections 2024-03-23T13:19:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f77f80a20c0 0x7f77f80a4580 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f780c1052c0 0x7f780c1a5320 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f780c102910 0x7f780c19e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 --2- 172.21.15.7:0/1489205756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f780c0fffa0 0x7f780c19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 -- 172.21.15.7:0/1489205756 >> 172.21.15.7:0/1489205756 conn(0x7f780c0fbbe0 msgr2=0x7f780c100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 -- 172.21.15.7:0/1489205756 shutdown_connections 2024-03-23T13:19:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.757+0000 7f7814156640 1 -- 172.21.15.7:0/1489205756 wait complete. 2024-03-23T13:19:31.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.871+0000 7fd8e864e640 1 -- 172.21.15.7:0/3055528810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8e00ffde0 msgr2=0x7fd8e0100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.871+0000 7fd8e864e640 1 --2- 172.21.15.7:0/3055528810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8e00ffde0 0x7fd8e0100200 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009900 tx=0x7fd8dc02bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.871+0000 7fd8e864e640 1 -- 172.21.15.7:0/3055528810 shutdown_connections 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.871+0000 7fd8e864e640 1 --2- 172.21.15.7:0/3055528810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8e0101990 0x7fd8e0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.871+0000 7fd8e864e640 1 --2- 172.21.15.7:0/3055528810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8e0100fd0 0x7fd8e0101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.871+0000 7fd8e864e640 1 --2- 172.21.15.7:0/3055528810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8e00ffde0 0x7fd8e0100200 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.871+0000 7fd8e864e640 1 -- 172.21.15.7:0/3055528810 >> 172.21.15.7:0/3055528810 conn(0x7fd8e00fb350 msgr2=0x7fd8e00fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.871+0000 7fd8e864e640 1 -- 172.21.15.7:0/3055528810 shutdown_connections 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.871+0000 7fd8e864e640 1 -- 172.21.15.7:0/3055528810 wait complete. 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e864e640 1 Processor -- start 2024-03-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e864e640 1 -- start start 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e864e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8e00ffde0 0x7fd8e0103ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e864e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8e0100fd0 0x7fd8e0102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e864e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8e0101990 0x7fd8e0102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e864e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e01031a0 con 0x7fd8e0101990 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e864e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e0103310 con 0x7fd8e00ffde0 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e864e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e01a18b0 con 0x7fd8e0100fd0 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e6beb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8e0101990 0x7fd8e0102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e63ea640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8e00ffde0 0x7fd8e0103ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e6beb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8e0101990 0x7fd8e0102a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60826/0 (socket says 172.21.15.7:60826) 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e63ea640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8e00ffde0 0x7fd8e0103ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:36818/0 (socket says 172.21.15.7:36818) 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e6beb640 1 -- 172.21.15.7:0/3006913568 learned_addr learned my addr 172.21.15.7:0/3006913568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e5be9640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8e0100fd0 0x7fd8e0102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e63ea640 1 -- 172.21.15.7:0/3006913568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8e0100fd0 msgr2=0x7fd8e0102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e63ea640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8e0100fd0 0x7fd8e0102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e63ea640 1 -- 172.21.15.7:0/3006913568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8e0101990 msgr2=0x7fd8e0102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e63ea640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8e0101990 0x7fd8e0102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e63ea640 1 -- 172.21.15.7:0/3006913568 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d8009840 con 0x7fd8e00ffde0 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e6beb640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8e0101990 0x7fd8e0102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.872+0000 7fd8e63ea640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8e00ffde0 0x7fd8e0103ec0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc036040 tx=0x7fd8dc005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.873+0000 7fd8d77fe640 1 -- 172.21.15.7:0/3006913568 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0043d0 con 0x7fd8e00ffde0 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.873+0000 7fd8d77fe640 1 -- 172.21.15.7:0/3006913568 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8dc0052d0 con 0x7fd8e00ffde0 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.873+0000 7fd8e864e640 1 -- 172.21.15.7:0/3006913568 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8dc009560 con 0x7fd8e00ffde0 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.873+0000 7fd8d77fe640 1 -- 172.21.15.7:0/3006913568 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc003cd0 con 0x7fd8e00ffde0 2024-03-23T13:19:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.873+0000 7fd8e864e640 1 -- 172.21.15.7:0/3006913568 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e01a1e80 con 0x7fd8e00ffde0 2024-03-23T13:19:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.874+0000 7fd8e864e640 1 -- 172.21.15.7:0/3006913568 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e010c060 con 0x7fd8e00ffde0 2024-03-23T13:19:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.874+0000 7fd8d77fe640 1 -- 172.21.15.7:0/3006913568 <== mon.2 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fd8dc005440 con 0x7fd8e00ffde0 2024-03-23T13:19:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.875+0000 7fd8d77fe640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd8cc0a20c0 0x7fd8cc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.875+0000 7fd8e5be9640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd8cc0a20c0 0x7fd8cc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.875+0000 7fd8d77fe640 1 -- 172.21.15.7:0/3006913568 <== mon.2 v2:172.21.15.130:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd8dc0efda0 con 0x7fd8e00ffde0 2024-03-23T13:19:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.875+0000 7fd8e5be9640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd8cc0a20c0 0x7fd8cc0a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0103c30 tx=0x7fd8d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:31.880+0000 7fd8d77fe640 1 -- 172.21.15.7:0/3006913568 <== mon.2 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd8dc0b8d20 con 0x7fd8e00ffde0 2024-03-23T13:19:32.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:31 smithi007 ceph-mon[32184]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:19:32.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:31 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1713928145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:32.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:31 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1732901879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:32.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:31 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2387125517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:32.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:31 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1489205756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.103+0000 7fd8e864e640 1 -- 172.21.15.7:0/3006913568 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8e01a2010 con 0x7fd8e00ffde0 2024-03-23T13:19:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.104+0000 7fd8d77fe640 1 -- 172.21.15.7:0/3006913568 <== mon.2 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11917 (secure 0 0 0) 0x7fd8dc0bc3e0 con 0x7fd8e00ffde0 2024-03-23T13:19:32.105 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:32.105 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T13:17:40.597309+0000","modified":"2024-03-23T13:17:48.259350+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24475,"mds_3":24470,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6841/3415570481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3415570481},{"type":"v1","addr":"172.21.15.7:6841","nonce":3415570481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/791500055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":791500055},{"type":"v1","addr":"172.21.15.130:6839","nonce":791500055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24470,24463,24475,24503,24496]},"id":1}]} 2024-03-23T13:19:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-23T13:19:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 -- 172.21.15.7:0/3006913568 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd8cc0a20c0 msgr2=0x7fd8cc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd8cc0a20c0 0x7fd8cc0a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0103c30 tx=0x7fd8d0005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 -- 172.21.15.7:0/3006913568 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8e00ffde0 msgr2=0x7fd8e0103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8e00ffde0 0x7fd8e0103ec0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc036040 tx=0x7fd8dc005e50 comp rx=0 tx=0).stop 2024-03-23T13:19:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 -- 172.21.15.7:0/3006913568 shutdown_connections 2024-03-23T13:19:32.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd8cc0a20c0 0x7fd8cc0a4580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8e0101990 0x7fd8e0102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8e0100fd0 0x7fd8e0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 --2- 172.21.15.7:0/3006913568 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8e00ffde0 0x7fd8e0103ec0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 -- 172.21.15.7:0/3006913568 >> 172.21.15.7:0/3006913568 conn(0x7fd8e00fb350 msgr2=0x7fd8e0108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:32.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 -- 172.21.15.7:0/3006913568 shutdown_connections 2024-03-23T13:19:32.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.106+0000 7fd8e864e640 1 -- 172.21.15.7:0/3006913568 wait complete. 2024-03-23T13:19:32.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:19:32.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:31 smithi130 ceph-mon[36225]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:19:32.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:31 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1713928145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:32.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:31 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1732901879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:32.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:31 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2387125517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:32.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:31 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1489205756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.220+0000 7f4229bdd640 1 -- 172.21.15.7:0/2857442069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42241021f0 msgr2=0x7f4224108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.220+0000 7f4229bdd640 1 --2- 172.21.15.7:0/2857442069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42241021f0 0x7f4224108a80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c02c0f0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.220+0000 7f4229bdd640 1 -- 172.21.15.7:0/2857442069 shutdown_connections 2024-03-23T13:19:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.220+0000 7f4229bdd640 1 --2- 172.21.15.7:0/2857442069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42241021f0 0x7f4224108a80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.220+0000 7f4229bdd640 1 --2- 172.21.15.7:0/2857442069 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4224101830 0x7f4224101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.220+0000 7f4229bdd640 1 --2- 172.21.15.7:0/2857442069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4224100640 0x7f4224100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.220+0000 7f4229bdd640 1 -- 172.21.15.7:0/2857442069 >> 172.21.15.7:0/2857442069 conn(0x7f42240fbbf0 msgr2=0x7f42240fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.220+0000 7f4229bdd640 1 -- 172.21.15.7:0/2857442069 shutdown_connections 2024-03-23T13:19:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.220+0000 7f4229bdd640 1 -- 172.21.15.7:0/2857442069 wait complete. 2024-03-23T13:19:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f4229bdd640 1 Processor -- start 2024-03-23T13:19:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f4229bdd640 1 -- start start 2024-03-23T13:19:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f4229bdd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224100640 0x7f42241045d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f4229bdd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4224101830 0x7f4224102c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f4229bdd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42241021f0 0x7f4224103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f4229bdd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42241038b0 con 0x7f4224100640 2024-03-23T13:19:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f4229bdd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4224103a20 con 0x7f4224101830 2024-03-23T13:19:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f4229bdd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42241a17e0 con 0x7f42241021f0 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f42237fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224100640 0x7f42241045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f42237fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224100640 0x7f42241045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60846/0 (socket says 172.21.15.7:60846) 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f42237fe640 1 -- 172.21.15.7:0/3367557836 learned_addr learned my addr 172.21.15.7:0/3367557836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f4223fff640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42241021f0 0x7f4224103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.221+0000 7f4222ffd640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4224101830 0x7f4224102c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f42237fe640 1 -- 172.21.15.7:0/3367557836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42241021f0 msgr2=0x7f4224103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f42237fe640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42241021f0 0x7f4224103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f42237fe640 1 -- 172.21.15.7:0/3367557836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4224101830 msgr2=0x7f4224102c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f42237fe640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4224101830 0x7f4224102c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f42237fe640 1 -- 172.21.15.7:0/3367557836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4214009490 con 0x7f4224100640 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f4222ffd640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4224101830 0x7f4224102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f42237fe640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224100640 0x7f42241045d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f421400e9f0 tx=0x7f421400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:32.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f4220ff9640 1 -- 172.21.15.7:0/3367557836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421400cde0 con 0x7f4224100640 2024-03-23T13:19:32.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f4220ff9640 1 -- 172.21.15.7:0/3367557836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4214010910 con 0x7f4224100640 2024-03-23T13:19:32.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f4229bdd640 1 -- 172.21.15.7:0/3367557836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c009900 con 0x7f4224100640 2024-03-23T13:19:32.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f4220ff9640 1 -- 172.21.15.7:0/3367557836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214018950 con 0x7f4224100640 2024-03-23T13:19:32.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.222+0000 7f4229bdd640 1 -- 172.21.15.7:0/3367557836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42241a1e10 con 0x7f4224100640 2024-03-23T13:19:32.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.223+0000 7f4229bdd640 1 -- 172.21.15.7:0/3367557836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f422410c020 con 0x7f4224100640 2024-03-23T13:19:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.224+0000 7f4220ff9640 1 -- 172.21.15.7:0/3367557836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f4214010a80 con 0x7f4224100640 2024-03-23T13:19:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.224+0000 7f4220ff9640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f420c0a20c0 0x7f420c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.225+0000 7f4222ffd640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f420c0a20c0 0x7f420c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.225+0000 7f4220ff9640 1 -- 172.21.15.7:0/3367557836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f4214014070 con 0x7f4224100640 2024-03-23T13:19:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.225+0000 7f4222ffd640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f420c0a20c0 0x7f420c0a4580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4224104340 tx=0x7f4218005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:32.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.229+0000 7f4220ff9640 1 -- 172.21.15.7:0/3367557836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4214097280 con 0x7f4224100640 2024-03-23T13:19:32.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:31 smithi155 ceph-mon[36157]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:19:32.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:31 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1713928145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:32.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:31 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1732901879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:32.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:31 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2387125517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:32.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:31 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1489205756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:32.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.415+0000 7f4229bdd640 1 -- 172.21.15.7:0/3367557836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f422410c1e0 con 0x7f4224100640 2024-03-23T13:19:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.417+0000 7f4220ff9640 1 -- 172.21.15.7:0/3367557836 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22558 (secure 0 0 0) 0x7f421409a940 con 0x7f4224100640 2024-03-23T13:19:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:32.418 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:18:43.667397+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:19:32.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 -- 172.21.15.7:0/3367557836 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f420c0a20c0 msgr2=0x7f420c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f420c0a20c0 0x7f420c0a4580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4224104340 tx=0x7f4218005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:32.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 -- 172.21.15.7:0/3367557836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224100640 msgr2=0x7f42241045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224100640 0x7f42241045d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f421400e9f0 tx=0x7f421400eeb0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 -- 172.21.15.7:0/3367557836 shutdown_connections 2024-03-23T13:19:32.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f420c0a20c0 0x7f420c0a4580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42241021f0 0x7f4224103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4224101830 0x7f4224102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 --2- 172.21.15.7:0/3367557836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224100640 0x7f42241045d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 -- 172.21.15.7:0/3367557836 >> 172.21.15.7:0/3367557836 conn(0x7f42240fbbf0 msgr2=0x7f42240fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 -- 172.21.15.7:0/3367557836 shutdown_connections 2024-03-23T13:19:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.419+0000 7f4229bdd640 1 -- 172.21.15.7:0/3367557836 wait complete. 2024-03-23T13:19:32.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T13:13:08.186719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T13:17:38.843887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T13:17:39.788489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T13:19:32.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T13:19:32.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.536+0000 7f505080b640 1 -- 172.21.15.7:0/1521262473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c0ffdb0 msgr2=0x7f504c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.536+0000 7f505080b640 1 --2- 172.21.15.7:0/1521262473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c0ffdb0 0x7f504c1001d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5044009900 tx=0x7f504402bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 -- 172.21.15.7:0/1521262473 shutdown_connections 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 --2- 172.21.15.7:0/1521262473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504c101960 0x7f504c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 --2- 172.21.15.7:0/1521262473 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f504c100fa0 0x7f504c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 --2- 172.21.15.7:0/1521262473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c0ffdb0 0x7f504c1001d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 -- 172.21.15.7:0/1521262473 >> 172.21.15.7:0/1521262473 conn(0x7f504c0fb320 msgr2=0x7f504c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 -- 172.21.15.7:0/1521262473 shutdown_connections 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 -- 172.21.15.7:0/1521262473 wait complete. 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 Processor -- start 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 -- start start 2024-03-23T13:19:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504c0ffdb0 0x7f504c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c100fa0 0x7f504c196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f504c101960 0x7f504c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f504c19e1c0 con 0x7f504c100fa0 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f504c19e330 con 0x7f504c101960 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.537+0000 7f505080b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f504c19e4f0 con 0x7f504c0ffdb0 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f5049d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c100fa0 0x7f504c196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f5049d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c100fa0 0x7f504c196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60874/0 (socket says 172.21.15.7:60874) 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f5049d9b640 1 -- 172.21.15.7:0/1546828612 learned_addr learned my addr 172.21.15.7:0/1546828612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f504a59c640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504c0ffdb0 0x7f504c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f504ad9d640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f504c101960 0x7f504c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f5049d9b640 1 -- 172.21.15.7:0/1546828612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504c0ffdb0 msgr2=0x7f504c196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f5049d9b640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504c0ffdb0 0x7f504c196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f5049d9b640 1 -- 172.21.15.7:0/1546828612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f504c101960 msgr2=0x7f504c19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f5049d9b640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f504c101960 0x7f504c19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f5049d9b640 1 -- 172.21.15.7:0/1546828612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5044009560 con 0x7f504c100fa0 2024-03-23T13:19:32.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f5049d9b640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c100fa0 0x7f504c196c60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f503400eb00 tx=0x7f5034009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:32.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f50437fe640 1 -- 172.21.15.7:0/1546828612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034013070 con 0x7f504c100fa0 2024-03-23T13:19:32.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f50437fe640 1 -- 172.21.15.7:0/1546828612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f503400bc30 con 0x7f504c100fa0 2024-03-23T13:19:32.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f505080b640 1 -- 172.21.15.7:0/1546828612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c108b50 con 0x7f504c100fa0 2024-03-23T13:19:32.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f50437fe640 1 -- 172.21.15.7:0/1546828612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50340166a0 con 0x7f504c100fa0 2024-03-23T13:19:32.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.538+0000 7f505080b640 1 -- 172.21.15.7:0/1546828612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c109060 con 0x7f504c100fa0 2024-03-23T13:19:32.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.539+0000 7f505080b640 1 -- 172.21.15.7:0/1546828612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c068950 con 0x7f504c100fa0 2024-03-23T13:19:32.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.540+0000 7f50437fe640 1 -- 172.21.15.7:0/1546828612 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f5034017980 con 0x7f504c100fa0 2024-03-23T13:19:32.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.541+0000 7f50437fe640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f50380a20c0 0x7f50380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.541+0000 7f504a59c640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f50380a20c0 0x7f50380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.541+0000 7f50437fe640 1 -- 172.21.15.7:0/1546828612 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f50340cf410 con 0x7f504c100fa0 2024-03-23T13:19:32.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.541+0000 7f504a59c640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f50380a20c0 0x7f50380a4580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5044005ab0 tx=0x7f5044005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.545+0000 7f50437fe640 1 -- 172.21.15.7:0/1546828612 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f50340984a0 con 0x7f504c100fa0 2024-03-23T13:19:32.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.726+0000 7f505080b640 1 -- 172.21.15.7:0/1546828612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f504c106ce0 con 0x7f504c100fa0 2024-03-23T13:19:32.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.727+0000 7f50437fe640 1 -- 172.21.15.7:0/1546828612 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f503409bb60 con 0x7f504c100fa0 2024-03-23T13:19:32.728 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-23T13:19:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 -- 172.21.15.7:0/1546828612 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f50380a20c0 msgr2=0x7f50380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f50380a20c0 0x7f50380a4580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5044005ab0 tx=0x7f5044005fd0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 -- 172.21.15.7:0/1546828612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c100fa0 msgr2=0x7f504c196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c100fa0 0x7f504c196c60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f503400eb00 tx=0x7f5034009f20 comp rx=0 tx=0).stop 2024-03-23T13:19:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 -- 172.21.15.7:0/1546828612 shutdown_connections 2024-03-23T13:19:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f50380a20c0 0x7f50380a4580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f504c101960 0x7f504c19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c100fa0 0x7f504c196c60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 --2- 172.21.15.7:0/1546828612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504c0ffdb0 0x7f504c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 -- 172.21.15.7:0/1546828612 >> 172.21.15.7:0/1546828612 conn(0x7f504c0fb320 msgr2=0x7f504c104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 -- 172.21.15.7:0/1546828612 shutdown_connections 2024-03-23T13:19:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.729+0000 7f505080b640 1 -- 172.21.15.7:0/1546828612 wait complete. 2024-03-23T13:19:32.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T13:19:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.846+0000 7f66d2664640 1 -- 172.21.15.7:0/4230466220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66cc101730 msgr2=0x7f66cc101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.846+0000 7f66d2664640 1 --2- 172.21.15.7:0/4230466220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66cc101730 0x7f66cc101b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009900 tx=0x7f66bc02bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.846+0000 7f66d2664640 1 -- 172.21.15.7:0/4230466220 shutdown_connections 2024-03-23T13:19:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.846+0000 7f66d2664640 1 --2- 172.21.15.7:0/4230466220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc1032e0 0x7f66cc109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.846+0000 7f66d2664640 1 --2- 172.21.15.7:0/4230466220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66cc102920 0x7f66cc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.846+0000 7f66d2664640 1 --2- 172.21.15.7:0/4230466220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66cc101730 0x7f66cc101b50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.846+0000 7f66d2664640 1 -- 172.21.15.7:0/4230466220 >> 172.21.15.7:0/4230466220 conn(0x7f66cc0fcca0 msgr2=0x7f66cc0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.846+0000 7f66d2664640 1 -- 172.21.15.7:0/4230466220 shutdown_connections 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.846+0000 7f66d2664640 1 -- 172.21.15.7:0/4230466220 wait complete. 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66d2664640 1 Processor -- start 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66d2664640 1 -- start start 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66d2664640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66cc101730 0x7f66cc1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66d2664640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66cc102920 0x7f66cc197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66d2664640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc1032e0 0x7f66cc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66d2664640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc19f2c0 con 0x7f66cc101730 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66d2664640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc19f430 con 0x7f66cc102920 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66d2664640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc19f5f0 con 0x7f66cc1032e0 2024-03-23T13:19:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66cc101730 0x7f66cc1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66cc101730 0x7f66cc1977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60894/0 (socket says 172.21.15.7:60894) 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66cbfff640 1 -- 172.21.15.7:0/233257893 learned_addr learned my addr 172.21.15.7:0/233257893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66cb7fe640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66cc102920 0x7f66cc197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66d0c01640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc1032e0 0x7f66cc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66cbfff640 1 -- 172.21.15.7:0/233257893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc1032e0 msgr2=0x7f66cc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66cbfff640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc1032e0 0x7f66cc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66cbfff640 1 -- 172.21.15.7:0/233257893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66cc102920 msgr2=0x7f66cc197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66cbfff640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66cc102920 0x7f66cc197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66cbfff640 1 -- 172.21.15.7:0/233257893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc009560 con 0x7f66cc101730 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.847+0000 7f66d0c01640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc1032e0 0x7f66cc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.848+0000 7f66cbfff640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66cc101730 0x7f66cc1977d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f66bc005ab0 tx=0x7f66bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.848+0000 7f66c97fa640 1 -- 172.21.15.7:0/233257893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc004aa0 con 0x7f66cc101730 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.848+0000 7f66d2664640 1 -- 172.21.15.7:0/233257893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66cc19f870 con 0x7f66cc101730 2024-03-23T13:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.848+0000 7f66c97fa640 1 -- 172.21.15.7:0/233257893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66bc0052d0 con 0x7f66cc101730 2024-03-23T13:19:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.848+0000 7f66c97fa640 1 -- 172.21.15.7:0/233257893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc003e80 con 0x7f66cc101730 2024-03-23T13:19:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.848+0000 7f66d2664640 1 -- 172.21.15.7:0/233257893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66cc19fd80 con 0x7f66cc101730 2024-03-23T13:19:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.849+0000 7f66d2664640 1 -- 172.21.15.7:0/233257893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66cc108ad0 con 0x7f66cc101730 2024-03-23T13:19:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.850+0000 7f66c97fa640 1 -- 172.21.15.7:0/233257893 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f66bc005000 con 0x7f66cc101730 2024-03-23T13:19:32.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.850+0000 7f66c97fa640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f66b80a20c0 0x7f66b80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:32.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.851+0000 7f66cb7fe640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f66b80a20c0 0x7f66b80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:32.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.851+0000 7f66c97fa640 1 -- 172.21.15.7:0/233257893 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f66bc0f0760 con 0x7f66cc101730 2024-03-23T13:19:32.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.851+0000 7f66cb7fe640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f66b80a20c0 0x7f66b80a4580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66cc198ce0 tx=0x7f66b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:32.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:32.854+0000 7f66c97fa640 1 -- 172.21.15.7:0/233257893 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f66bc0b9740 con 0x7f66cc101730 2024-03-23T13:19:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.037+0000 7f66d2664640 1 -- 172.21.15.7:0/233257893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f66cc10d2b0 con 0x7f66cc101730 2024-03-23T13:19:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.037+0000 7f66c97fa640 1 -- 172.21.15.7:0/233257893 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f66cc10d2b0 con 0x7f66cc101730 2024-03-23T13:19:33.038 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-23T13:19:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 -- 172.21.15.7:0/233257893 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f66b80a20c0 msgr2=0x7f66b80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f66b80a20c0 0x7f66b80a4580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66cc198ce0 tx=0x7f66b4005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 -- 172.21.15.7:0/233257893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66cc101730 msgr2=0x7f66cc1977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66cc101730 0x7f66cc1977d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f66bc005ab0 tx=0x7f66bc004270 comp rx=0 tx=0).stop 2024-03-23T13:19:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 -- 172.21.15.7:0/233257893 shutdown_connections 2024-03-23T13:19:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f66b80a20c0 0x7f66b80a4580 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc1032e0 0x7f66cc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66cc102920 0x7f66cc197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 --2- 172.21.15.7:0/233257893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66cc101730 0x7f66cc1977d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 -- 172.21.15.7:0/233257893 >> 172.21.15.7:0/233257893 conn(0x7f66cc0fcca0 msgr2=0x7f66cc105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 -- 172.21.15.7:0/233257893 shutdown_connections 2024-03-23T13:19:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.039+0000 7f66d2664640 1 -- 172.21.15.7:0/233257893 wait complete. 2024-03-23T13:19:33.050 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:32 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3006913568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:33.050 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:32 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3367557836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:33.050 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:32 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1546828612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:33.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T13:19:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.156+0000 7fddd39d7640 1 -- 172.21.15.7:0/1820561755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc104ac0 msgr2=0x7fddcc106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.156+0000 7fddd39d7640 1 --2- 172.21.15.7:0/1820561755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc104ac0 0x7fddcc106eb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009900 tx=0x7fddbc02bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.156+0000 7fddd39d7640 1 -- 172.21.15.7:0/1820561755 shutdown_connections 2024-03-23T13:19:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.156+0000 7fddd39d7640 1 --2- 172.21.15.7:0/1820561755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddcc1073f0 0x7fddcc109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.156+0000 7fddd39d7640 1 --2- 172.21.15.7:0/1820561755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc104ac0 0x7fddcc106eb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.156+0000 7fddd39d7640 1 --2- 172.21.15.7:0/1820561755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddcc102190 0x7fddcc104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.156+0000 7fddd39d7640 1 -- 172.21.15.7:0/1820561755 >> 172.21.15.7:0/1820561755 conn(0x7fddcc0fbb80 msgr2=0x7fddcc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.156+0000 7fddd39d7640 1 -- 172.21.15.7:0/1820561755 shutdown_connections 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.156+0000 7fddd39d7640 1 -- 172.21.15.7:0/1820561755 wait complete. 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd39d7640 1 Processor -- start 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd39d7640 1 -- start start 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd39d7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddcc102190 0x7fddcc1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd39d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc104ac0 0x7fddcc197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd39d7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddcc1073f0 0x7fddcc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd39d7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc19f2e0 con 0x7fddcc104ac0 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd39d7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc19f450 con 0x7fddcc1073f0 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd39d7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc19f610 con 0x7fddcc102190 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd0f72640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc104ac0 0x7fddcc197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd0f72640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc104ac0 0x7fddcc197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60918/0 (socket says 172.21.15.7:60918) 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd0f72640 1 -- 172.21.15.7:0/1898924519 learned_addr learned my addr 172.21.15.7:0/1898924519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd1773640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddcc102190 0x7fddcc1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd1f74640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddcc1073f0 0x7fddcc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd0f72640 1 -- 172.21.15.7:0/1898924519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddcc102190 msgr2=0x7fddcc1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd0f72640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddcc102190 0x7fddcc1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd0f72640 1 -- 172.21.15.7:0/1898924519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddcc1073f0 msgr2=0x7fddcc19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd0f72640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddcc1073f0 0x7fddcc19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd0f72640 1 -- 172.21.15.7:0/1898924519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc009560 con 0x7fddcc104ac0 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddd0f72640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc104ac0 0x7fddcc197d30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fddbc005ab0 tx=0x7fddbc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.157+0000 7fddc27fc640 1 -- 172.21.15.7:0/1898924519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0043d0 con 0x7fddcc104ac0 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.158+0000 7fddd39d7640 1 -- 172.21.15.7:0/1898924519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddcc19f890 con 0x7fddcc104ac0 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.158+0000 7fddc27fc640 1 -- 172.21.15.7:0/1898924519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddbc0052d0 con 0x7fddcc104ac0 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.158+0000 7fddc27fc640 1 -- 172.21.15.7:0/1898924519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc003d40 con 0x7fddcc104ac0 2024-03-23T13:19:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.158+0000 7fddd39d7640 1 -- 172.21.15.7:0/1898924519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddcc19fdf0 con 0x7fddcc104ac0 2024-03-23T13:19:33.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.159+0000 7fddd39d7640 1 -- 172.21.15.7:0/1898924519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddcc0fd670 con 0x7fddcc104ac0 2024-03-23T13:19:33.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.160+0000 7fddc27fc640 1 -- 172.21.15.7:0/1898924519 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fddbc005440 con 0x7fddcc104ac0 2024-03-23T13:19:33.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.160+0000 7fddc27fc640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fddb80a20c0 0x7fddb80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.160+0000 7fddd1773640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fddb80a20c0 0x7fddb80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.160+0000 7fddc27fc640 1 -- 172.21.15.7:0/1898924519 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fddbc0efdb0 con 0x7fddcc104ac0 2024-03-23T13:19:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.161+0000 7fddd1773640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fddb80a20c0 0x7fddb80a4580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009bd0 tx=0x7fddc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:33.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.165+0000 7fddc27fc640 1 -- 172.21.15.7:0/1898924519 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fddbc0b8d90 con 0x7fddcc104ac0 2024-03-23T13:19:33.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:32 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3006913568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:33.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:32 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3367557836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:33.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:32 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1546828612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:33.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:32 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3006913568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:33.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:32 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3367557836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:33.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:32 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1546828612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:19:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.350+0000 7fddd39d7640 1 -- 172.21.15.7:0/1898924519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fddcc02d160 con 0x7fddcc104ac0 2024-03-23T13:19:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.350+0000 7fddc27fc640 1 -- 172.21.15.7:0/1898924519 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fddbc0bc450 con 0x7fddcc104ac0 2024-03-23T13:19:33.351 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.352+0000 7fddd39d7640 1 -- 172.21.15.7:0/1898924519 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fddb80a20c0 msgr2=0x7fddb80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.352+0000 7fddd39d7640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fddb80a20c0 0x7fddb80a4580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009bd0 tx=0x7fddc8005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.352+0000 7fddd39d7640 1 -- 172.21.15.7:0/1898924519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc104ac0 msgr2=0x7fddcc197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.352+0000 7fddd39d7640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc104ac0 0x7fddcc197d30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fddbc005ab0 tx=0x7fddbc005e50 comp rx=0 tx=0).stop 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.352+0000 7fddd39d7640 1 -- 172.21.15.7:0/1898924519 shutdown_connections 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.352+0000 7fddd39d7640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fddb80a20c0 0x7fddb80a4580 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.352+0000 7fddd39d7640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddcc1073f0 0x7fddcc19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.352+0000 7fddd39d7640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddcc104ac0 0x7fddcc197d30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.352+0000 7fddd39d7640 1 --2- 172.21.15.7:0/1898924519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddcc102190 0x7fddcc1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.352+0000 7fddd39d7640 1 -- 172.21.15.7:0/1898924519 >> 172.21.15.7:0/1898924519 conn(0x7fddcc0fbb80 msgr2=0x7fddcc102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.353+0000 7fddd39d7640 1 -- 172.21.15.7:0/1898924519 shutdown_connections 2024-03-23T13:19:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.353+0000 7fddd39d7640 1 -- 172.21.15.7:0/1898924519 wait complete. 2024-03-23T13:19:33.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.467+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3362405847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d18102930 msgr2=0x7f2d18102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.467+0000 7f2d1d0c3640 1 --2- 172.21.15.7:0/3362405847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d18102930 0x7f2d18102db0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009900 tx=0x7f2d0c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.468+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3362405847 shutdown_connections 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.468+0000 7f2d1d0c3640 1 --2- 172.21.15.7:0/3362405847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d181032f0 0x7f2d18109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.468+0000 7f2d1d0c3640 1 --2- 172.21.15.7:0/3362405847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d18102930 0x7f2d18102db0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.468+0000 7f2d1d0c3640 1 --2- 172.21.15.7:0/3362405847 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d18101740 0x7f2d18101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.468+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3362405847 >> 172.21.15.7:0/3362405847 conn(0x7f2d180fccf0 msgr2=0x7f2d180ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.468+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3362405847 shutdown_connections 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.468+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3362405847 wait complete. 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.468+0000 7f2d1d0c3640 1 Processor -- start 2024-03-23T13:19:33.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.468+0000 7f2d1d0c3640 1 -- start start 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1d0c3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d18101740 0x7f2d180754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1d0c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d18102930 0x7f2d18073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1d0c3640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d181032f0 0x7f2d18074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1d0c3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1819b610 con 0x7f2d18102930 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1d0c3640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18074580 con 0x7f2d181032f0 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1d0c3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18074740 con 0x7f2d18101740 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d18102930 0x7f2d18073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d18102930 0x7f2d18073b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60942/0 (socket says 172.21.15.7:60942) 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d16d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d18101740 0x7f2d180754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1659c640 1 -- 172.21.15.7:0/3949795092 learned_addr learned my addr 172.21.15.7:0/3949795092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1759e640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d181032f0 0x7f2d18074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1659c640 1 -- 172.21.15.7:0/3949795092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d18101740 msgr2=0x7f2d180754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1659c640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d18101740 0x7f2d180754c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1659c640 1 -- 172.21.15.7:0/3949795092 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d181032f0 msgr2=0x7f2d18074040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1659c640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d181032f0 0x7f2d18074040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1659c640 1 -- 172.21.15.7:0/3949795092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c009560 con 0x7f2d18102930 2024-03-23T13:19:33.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1659c640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d18102930 0x7f2d18073b00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c006010 tx=0x7f2d0c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:33.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2cfffff640 1 -- 172.21.15.7:0/3949795092 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0044c0 con 0x7f2d18102930 2024-03-23T13:19:33.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3949795092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d18074960 con 0x7f2d18102930 2024-03-23T13:19:33.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2cfffff640 1 -- 172.21.15.7:0/3949795092 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d0c0052d0 con 0x7f2d18102930 2024-03-23T13:19:33.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2cfffff640 1 -- 172.21.15.7:0/3949795092 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c003e10 con 0x7f2d18102930 2024-03-23T13:19:33.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.469+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3949795092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d181a63d0 con 0x7f2d18102930 2024-03-23T13:19:33.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.470+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3949795092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d18108a80 con 0x7f2d18102930 2024-03-23T13:19:33.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.471+0000 7f2cfffff640 1 -- 172.21.15.7:0/3949795092 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f2d0c005440 con 0x7f2d18102930 2024-03-23T13:19:33.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.472+0000 7f2cfffff640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d000a20c0 0x7f2d000a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.472+0000 7f2d16d9d640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d000a20c0 0x7f2d000a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.472+0000 7f2cfffff640 1 -- 172.21.15.7:0/3949795092 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2d0c0f0590 con 0x7f2d18102930 2024-03-23T13:19:33.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.472+0000 7f2d16d9d640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d000a20c0 0x7f2d000a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2d08005fd0 tx=0x7f2d0800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:33.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.476+0000 7f2cfffff640 1 -- 172.21.15.7:0/3949795092 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2d0c0b9570 con 0x7f2d18102930 2024-03-23T13:19:33.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.697+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3949795092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d1810d150 con 0x7f2d18102930 2024-03-23T13:19:33.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.697+0000 7f2cfffff640 1 -- 172.21.15.7:0/3949795092 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11917 (secure 0 0 0) 0x7f2d0c0bcc30 con 0x7f2d18102930 2024-03-23T13:19:33.698 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:33.698 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T13:17:40.597309+0000","modified":"2024-03-23T13:17:48.259350+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24475,"mds_3":24470,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6841/3415570481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3415570481},{"type":"v1","addr":"172.21.15.7:6841","nonce":3415570481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/791500055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":791500055},{"type":"v1","addr":"172.21.15.130:6839","nonce":791500055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24470,24463,24475,24503,24496]},"id":1}]} 2024-03-23T13:19:33.699 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-23T13:19:33.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.699+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3949795092 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d000a20c0 msgr2=0x7f2d000a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.699+0000 7f2d1d0c3640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d000a20c0 0x7f2d000a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2d08005fd0 tx=0x7f2d0800b410 comp rx=0 tx=0).stop 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.699+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3949795092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d18102930 msgr2=0x7f2d18073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.699+0000 7f2d1d0c3640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d18102930 0x7f2d18073b00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c006010 tx=0x7f2d0c0040c0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.700+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3949795092 shutdown_connections 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.700+0000 7f2d1d0c3640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f2d000a20c0 0x7f2d000a4580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.700+0000 7f2d1d0c3640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d181032f0 0x7f2d18074040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.700+0000 7f2d1d0c3640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d18102930 0x7f2d18073b00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.700+0000 7f2d1d0c3640 1 --2- 172.21.15.7:0/3949795092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d18101740 0x7f2d180754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.700+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3949795092 >> 172.21.15.7:0/3949795092 conn(0x7f2d180fccf0 msgr2=0x7f2d18105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.700+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3949795092 shutdown_connections 2024-03-23T13:19:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.700+0000 7f2d1d0c3640 1 -- 172.21.15.7:0/3949795092 wait complete. 2024-03-23T13:19:33.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:19:33.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.814+0000 7fb294c57640 1 -- 172.21.15.7:0/2082726881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb290100650 msgr2=0x7fb290100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.814+0000 7fb294c57640 1 --2- 172.21.15.7:0/2082726881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb290100650 0x7fb290100a70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb288009900 tx=0x7fb28802bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:33.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.814+0000 7fb294c57640 1 -- 172.21.15.7:0/2082726881 shutdown_connections 2024-03-23T13:19:33.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.814+0000 7fb294c57640 1 --2- 172.21.15.7:0/2082726881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb290102200 0x7fb290108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.814+0000 7fb294c57640 1 --2- 172.21.15.7:0/2082726881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb290101840 0x7fb290101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.814+0000 7fb294c57640 1 --2- 172.21.15.7:0/2082726881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb290100650 0x7fb290100a70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.814+0000 7fb294c57640 1 -- 172.21.15.7:0/2082726881 >> 172.21.15.7:0/2082726881 conn(0x7fb2900fbbc0 msgr2=0x7fb2900fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.814+0000 7fb294c57640 1 -- 172.21.15.7:0/2082726881 shutdown_connections 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.814+0000 7fb294c57640 1 -- 172.21.15.7:0/2082726881 wait complete. 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.814+0000 7fb294c57640 1 Processor -- start 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb294c57640 1 -- start start 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb294c57640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb290100650 0x7fb290197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb294c57640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb290101840 0x7fb290197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb294c57640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb290102200 0x7fb29019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb294c57640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb29019f300 con 0x7fb290100650 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb294c57640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb29019f470 con 0x7fb290101840 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb294c57640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb29019f630 con 0x7fb290102200 2024-03-23T13:19:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb290100650 0x7fb290197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb290100650 0x7fb290197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60956/0 (socket says 172.21.15.7:60956) 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28e59c640 1 -- 172.21.15.7:0/1493458197 learned_addr learned my addr 172.21.15.7:0/1493458197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28dd9b640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb290101840 0x7fb290197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28ed9d640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb290102200 0x7fb29019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28e59c640 1 -- 172.21.15.7:0/1493458197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb290102200 msgr2=0x7fb29019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28e59c640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb290102200 0x7fb29019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28e59c640 1 -- 172.21.15.7:0/1493458197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb290101840 msgr2=0x7fb290197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28e59c640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb290101840 0x7fb290197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28e59c640 1 -- 172.21.15.7:0/1493458197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb288009560 con 0x7fb290100650 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb28e59c640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb290100650 0x7fb290197810 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb288005ab0 tx=0x7fb288005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb27f7fe640 1 -- 172.21.15.7:0/1493458197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2880043d0 con 0x7fb290100650 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb27f7fe640 1 -- 172.21.15.7:0/1493458197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2880052d0 con 0x7fb290100650 2024-03-23T13:19:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb294c57640 1 -- 172.21.15.7:0/1493458197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29019f8b0 con 0x7fb290100650 2024-03-23T13:19:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.815+0000 7fb27f7fe640 1 -- 172.21.15.7:0/1493458197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288003d40 con 0x7fb290100650 2024-03-23T13:19:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.816+0000 7fb294c57640 1 -- 172.21.15.7:0/1493458197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29019fdc0 con 0x7fb290100650 2024-03-23T13:19:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.816+0000 7fb294c57640 1 -- 172.21.15.7:0/1493458197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb290107970 con 0x7fb290100650 2024-03-23T13:19:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.817+0000 7fb27f7fe640 1 -- 172.21.15.7:0/1493458197 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fb288005440 con 0x7fb290100650 2024-03-23T13:19:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.818+0000 7fb27f7fe640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb2780a20c0 0x7fb2780a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.818+0000 7fb28dd9b640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb2780a20c0 0x7fb2780a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.818+0000 7fb27f7fe640 1 -- 172.21.15.7:0/1493458197 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb2880efdb0 con 0x7fb290100650 2024-03-23T13:19:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.819+0000 7fb28dd9b640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb2780a20c0 0x7fb2780a4580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb290198d20 tx=0x7fb284005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:33.823+0000 7fb27f7fe640 1 -- 172.21.15.7:0/1493458197 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb2880b8d90 con 0x7fb290100650 2024-03-23T13:19:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.009+0000 7fb294c57640 1 -- 172.21.15.7:0/1493458197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb29002d160 con 0x7fb290100650 2024-03-23T13:19:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.010+0000 7fb27f7fe640 1 -- 172.21.15.7:0/1493458197 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22558 (secure 0 0 0) 0x7fb2880bc450 con 0x7fb290100650 2024-03-23T13:19:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:19:34.012 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:18:43.667397+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:19:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 -- 172.21.15.7:0/1493458197 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb2780a20c0 msgr2=0x7fb2780a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb2780a20c0 0x7fb2780a4580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb290198d20 tx=0x7fb284005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 -- 172.21.15.7:0/1493458197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb290100650 msgr2=0x7fb290197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb290100650 0x7fb290197810 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb288005ab0 tx=0x7fb288005e50 comp rx=0 tx=0).stop 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 -- 172.21.15.7:0/1493458197 shutdown_connections 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb2780a20c0 0x7fb2780a4580 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb290102200 0x7fb29019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb290101840 0x7fb290197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 --2- 172.21.15.7:0/1493458197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb290100650 0x7fb290197810 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 -- 172.21.15.7:0/1493458197 >> 172.21.15.7:0/1493458197 conn(0x7fb2900fbbc0 msgr2=0x7fb290104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 -- 172.21.15.7:0/1493458197 shutdown_connections 2024-03-23T13:19:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.013+0000 7fb294c57640 1 -- 172.21.15.7:0/1493458197 wait complete. 2024-03-23T13:19:34.023 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:33 smithi007 ceph-mon[32184]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:19:34.024 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:33 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/233257893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:34.024 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:33 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1898924519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:34.024 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:33 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3949795092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:34.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-23T13:17:40.597309+0000', 'modified': '2024-03-23T13:17:48.259350+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24503, 'mds_1': 24496, 'mds_2': 24475, 'mds_3': 24470, 'mds_4': 24463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.7:6839/388542752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6838', 'nonce': 388542752}, {'type': 'v1', 'addr': '172.21.15.7:6839', 'nonce': 388542752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14649': {'gid': 14649, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.7:6841/3415570481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6840', 'nonce': 3415570481}, {'type': 'v1', 'addr': '172.21.15.7:6841', 'nonce': 3415570481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6833/3818242586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6832', 'nonce': 3818242586}, {'type': 'v1', 'addr': '172.21.15.130:6833', 'nonce': 3818242586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6833/1319723529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 1319723529}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 1319723529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6835/2599360099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6834', 'nonce': 2599360099}, {'type': 'v1', 'addr': '172.21.15.130:6835', 'nonce': 2599360099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.155:6835/1880391142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 1880391142}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 1880391142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.130:6837/877142007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 877142007}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 877142007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.155:6837/3999767587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 3999767587}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 3999767587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.155:6839/131946068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 131946068}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 131946068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.130:6839/791500055', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 791500055}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 791500055}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24503, 'qdb_cluster': [24470, 24463, 24475, 24503, 24496]} 2024-03-23T13:19:34.026 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-23T13:19:34.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-23T13:19:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.129+0000 7f90957dd640 1 -- 172.21.15.7:0/3399313955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90901006b0 msgr2=0x7f9090100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.129+0000 7f90957dd640 1 --2- 172.21.15.7:0/3399313955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90901006b0 0x7f9090100ad0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9088009900 tx=0x7f908802bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.129+0000 7f90957dd640 1 -- 172.21.15.7:0/3399313955 shutdown_connections 2024-03-23T13:19:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.129+0000 7f90957dd640 1 --2- 172.21.15.7:0/3399313955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9090102260 0x7f9090108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.129+0000 7f90957dd640 1 --2- 172.21.15.7:0/3399313955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90901018a0 0x7f9090101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.129+0000 7f90957dd640 1 --2- 172.21.15.7:0/3399313955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90901006b0 0x7f9090100ad0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.129+0000 7f90957dd640 1 -- 172.21.15.7:0/3399313955 >> 172.21.15.7:0/3399313955 conn(0x7f90900fbc20 msgr2=0x7f90900fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.129+0000 7f90957dd640 1 -- 172.21.15.7:0/3399313955 shutdown_connections 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.129+0000 7f90957dd640 1 -- 172.21.15.7:0/3399313955 wait complete. 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f90957dd640 1 Processor -- start 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f90957dd640 1 -- start start 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f90957dd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90901006b0 0x7f90901966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f90957dd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90901018a0 0x7f9090196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f90957dd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9090102260 0x7f909019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f90957dd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f909019e1e0 con 0x7f9090102260 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f90957dd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f909019e350 con 0x7f90901006b0 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f90957dd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f909019e510 con 0x7f90901018a0 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f908f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9090102260 0x7f909019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f908f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9090102260 0x7f909019dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60976/0 (socket says 172.21.15.7:60976) 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f908f7fe640 1 -- 172.21.15.7:0/354345762 learned_addr learned my addr 172.21.15.7:0/354345762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f908effd640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90901006b0 0x7f90901966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f908e7fc640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90901018a0 0x7f9090196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f908f7fe640 1 -- 172.21.15.7:0/354345762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90901018a0 msgr2=0x7f9090196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f908f7fe640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90901018a0 0x7f9090196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f908f7fe640 1 -- 172.21.15.7:0/354345762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90901006b0 msgr2=0x7f90901966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.130+0000 7f908f7fe640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90901006b0 0x7f90901966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.131+0000 7f908f7fe640 1 -- 172.21.15.7:0/354345762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9088009560 con 0x7f9090102260 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.131+0000 7f908f7fe640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9090102260 0x7f909019dca0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f908000eb10 tx=0x7f908000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.131+0000 7f9087fff640 1 -- 172.21.15.7:0/354345762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080004b20 con 0x7f9090102260 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.131+0000 7f9087fff640 1 -- 172.21.15.7:0/354345762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9080004c80 con 0x7f9090102260 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.131+0000 7f90957dd640 1 -- 172.21.15.7:0/354345762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909019e7f0 con 0x7f9090102260 2024-03-23T13:19:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.131+0000 7f9087fff640 1 -- 172.21.15.7:0/354345762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080010650 con 0x7f9090102260 2024-03-23T13:19:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.131+0000 7f90957dd640 1 -- 172.21.15.7:0/354345762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909019ed00 con 0x7f9090102260 2024-03-23T13:19:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.133+0000 7f9087fff640 1 -- 172.21.15.7:0/354345762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f9080005020 con 0x7f9090102260 2024-03-23T13:19:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.133+0000 7f90983b6740 1 Processor -- start 2024-03-23T13:19:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.133+0000 7f90983b6740 1 -- start start 2024-03-23T13:19:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f90983b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e329b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f90983b6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32a160 0x55aa9e32ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f90983b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32f2d0 0x55aa9e331760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f90983b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aa9e296180 con 0x55aa9e329770 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f90983b6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aa9e2936b0 con 0x55aa9e32a160 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f90983b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aa9e28ee90 con 0x55aa9e32f2d0 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e329b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f9087fff640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f907c0a20c0 0x7f907c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e329b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60986/0 (socket says 172.21.15.7:60986) 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908e7fc640 1 -- 172.21.15.7:0/3592690168 learned_addr learned my addr 172.21.15.7:0/3592690168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908effd640 1 --2- 172.21.15.7:0/3592690168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32a160 0x55aa9e32ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908e7fc640 1 --2- 172.21.15.7:0/3592690168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32f2d0 0x55aa9e331760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908f7fe640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f907c0a20c0 0x7f907c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f9087fff640 1 -- 172.21.15.7:0/354345762 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9080014070 con 0x7f9090102260 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908e7fc640 1 -- 172.21.15.7:0/3592690168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32f2d0 msgr2=0x55aa9e331760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908e7fc640 1 --2- 172.21.15.7:0/3592690168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32f2d0 0x55aa9e331760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908e7fc640 1 -- 172.21.15.7:0/3592690168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32a160 msgr2=0x55aa9e32ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908e7fc640 1 --2- 172.21.15.7:0/3592690168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32a160 0x55aa9e32ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908e7fc640 1 -- 172.21.15.7:0/3592690168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa9e24bd80 con 0x55aa9e329770 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908effd640 1 --2- 172.21.15.7:0/3592690168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32a160 0x55aa9e32ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908e7fc640 1 --2- 172.21.15.7:0/3592690168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e329b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f90780137f0 tx=0x7f90780097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93b58ba49918e8b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f908f7fe640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f907c0a20c0 0x7f907c0a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9080097380 tx=0x7f9080090400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f90857fa640 1 -- 172.21.15.7:0/3592690168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078013d70 con 0x55aa9e329770 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f90857fa640 1 -- 172.21.15.7:0/3592690168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9078005500 con 0x55aa9e329770 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.134+0000 7f90857fa640 1 -- 172.21.15.7:0/3592690168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078011a40 con 0x55aa9e329770 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.135+0000 7f90983b6740 1 -- 172.21.15.7:0/3592690168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 msgr2=0x55aa9e329b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.135+0000 7f90983b6740 1 --2- 172.21.15.7:0/3592690168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e329b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f90780137f0 tx=0x7f90780097a0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.135+0000 7f90983b6740 1 -- 172.21.15.7:0/3592690168 shutdown_connections 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.135+0000 7f90983b6740 1 --2- 172.21.15.7:0/3592690168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32f2d0 0x55aa9e331760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.135+0000 7f90983b6740 1 --2- 172.21.15.7:0/3592690168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32a160 0x55aa9e32ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.135+0000 7f90983b6740 1 --2- 172.21.15.7:0/3592690168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e329b90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.135+0000 7f90983b6740 1 -- 172.21.15.7:0/3592690168 >> 172.21.15.7:0/3592690168 conn(0x55aa9e324ce0 msgr2=0x55aa9e3250e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.135+0000 7f90983b6740 1 -- 172.21.15.7:0/3592690168 shutdown_connections 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.135+0000 7f90983b6740 1 -- 172.21.15.7:0/3592690168 wait complete. 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f90983b6740 1 -- start start 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f90983b6740 1 Processor -- start 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f90983b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e3336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f90983b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32a160 0x55aa9e333c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f90983b6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32f2d0 0x55aa9e33abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f90983b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aa9e233d50 con 0x55aa9e329770 2024-03-23T13:19:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f90983b6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aa9df62310 con 0x55aa9e32f2d0 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f90983b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aa9e2148f0 con 0x55aa9e32a160 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f90983b6740 10 client.0 _ll_register_callbacks cb 0x55aa9df9fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e3336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e3336e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60998/0 (socket says 172.21.15.7:60998) 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908e7fc640 1 -- 172.21.15.7:0/4114893855 learned_addr learned my addr 172.21.15.7:0/4114893855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908effd640 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32a160 0x55aa9e333c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908e7fc640 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32f2d0 0x55aa9e33abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908e7fc640 1 -- 172.21.15.7:0/4114893855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32a160 msgr2=0x55aa9e333c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908e7fc640 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32a160 0x55aa9e333c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908e7fc640 1 -- 172.21.15.7:0/4114893855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32f2d0 msgr2=0x55aa9e33abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908e7fc640 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32f2d0 0x55aa9e33abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908e7fc640 1 -- 172.21.15.7:0/4114893855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907801b040 con 0x55aa9e329770 2024-03-23T13:19:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f908e7fc640 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e3336e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9078013600 tx=0x7f9078005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1712bdd542c1bcb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.136+0000 7f90767fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90767fc640 1 -- 172.21.15.7:0/4114893855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078004040 con 0x55aa9e329770 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90767fc640 1 -- 172.21.15.7:0/4114893855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f90780041a0 con 0x55aa9e329770 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90983b6740 1 -- 172.21.15.7:0/4114893855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa9df8e6c0 con 0x55aa9e329770 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90767fc640 1 -- 172.21.15.7:0/4114893855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9078004360 con 0x55aa9e329770 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90983b6740 10 client.15267 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90983b6740 1 -- 172.21.15.7:0/4114893855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa9e28d510 con 0x55aa9e329770 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90767fc640 1 -- 172.21.15.7:0/4114893855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55aa9df8e6c0 con 0x55aa9e329770 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90767fc640 1 -- 172.21.15.7:0/4114893855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9078035070 con 0x55aa9e329770 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90983b6740 10 client.15267 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90983b6740 10 client.15267 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90983b6740 20 client.15267 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90983b6740 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55aa9e33fd70 0x55aa9e3411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90983b6740 4 client.15267 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f90983b6740 1 -- 172.21.15.7:0/4114893855 --> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa9dfdf270 con 0x55aa9e33fd70 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.137+0000 7f908effd640 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55aa9e33fd70 0x55aa9e3411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.138+0000 7f90767fc640 10 client.15267 ms_handle_connect on v2:172.21.15.155:6834/1880391142 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.138+0000 7f908effd640 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55aa9e33fd70 0x55aa9e3411e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ed0bfc57a7b49ea server_cookie=1c2f8a3d797bccd7 in_seq=0 out_seq=0 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.138+0000 7f90767fc640 1 -- 172.21.15.7:0/4114893855 <== mds.4 v2:172.21.15.155:6834/1880391142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55aa9dfdf270 con 0x55aa9e33fd70 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.138+0000 7f90767fc640 10 client.15267 handle_command_reply: tid=0 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.138+0000 7f90983b6740 1 client.15267 shutdown 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 1 -- 172.21.15.7:0/4114893855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 msgr2=0x55aa9e3336e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e3336e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9078013600 tx=0x7f9078005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 1 -- 172.21.15.7:0/4114893855 shutdown_connections 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32f2d0 0x55aa9e33abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32a160 0x55aa9e333c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55aa9e33fd70 0x55aa9e3411e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 1 --2- 172.21.15.7:0/4114893855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e329770 0x55aa9e3336e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90767fc640 0 client.15267 ms_handle_reset on v2:172.21.15.155:6834/1880391142 2024-03-23T13:19:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 1 -- 172.21.15.7:0/4114893855 >> 172.21.15.7:0/4114893855 conn(0x55aa9e324ce0 msgr2=0x55aa9e321ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 1 -- 172.21.15.7:0/4114893855 shutdown_connections 2024-03-23T13:19:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 1 -- 172.21.15.7:0/4114893855 wait complete. 2024-03-23T13:19:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.139+0000 7f90983b6740 20 client.15267 trim_cache size 0 max 16384 2024-03-23T13:19:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f90983b6740 1 Processor -- start 2024-03-23T13:19:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f90983b6740 1 -- start start 2024-03-23T13:19:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f90983b6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e329770 0x55aa9e3332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f90983b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f90983b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32f2d0 0x55aa9e333830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f90983b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aa9e331e20 con 0x55aa9e32a160 2024-03-23T13:19:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f90983b6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aa9e340150 con 0x55aa9e329770 2024-03-23T13:19:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f90983b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aa9dfdf270 con 0x55aa9e32f2d0 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f908effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f908effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32770/0 (socket says 172.21.15.7:32770) 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f908effd640 1 -- 172.21.15.7:0/4250845924 learned_addr learned my addr 172.21.15.7:0/4250845924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f908e7fc640 1 --2- 172.21.15.7:0/4250845924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e329770 0x55aa9e3332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f908e7fc640 1 --2- 172.21.15.7:0/4250845924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32f2d0 0x55aa9e333830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f908effd640 1 -- 172.21.15.7:0/4250845924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32f2d0 msgr2=0x55aa9e333830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f908effd640 1 --2- 172.21.15.7:0/4250845924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32f2d0 0x55aa9e333830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f908effd640 1 -- 172.21.15.7:0/4250845924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e329770 msgr2=0x55aa9e3332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f908effd640 1 --2- 172.21.15.7:0/4250845924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e329770 0x55aa9e3332f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.157+0000 7f908effd640 1 -- 172.21.15.7:0/4250845924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa9e229a80 con 0x55aa9e32a160 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f908effd640 1 --2- 172.21.15.7:0/4250845924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33abb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9088009230 tx=0x7f9088005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7271612a198051c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f9084ff9640 1 -- 172.21.15.7:0/4250845924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908804ab10 con 0x55aa9e32a160 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f9084ff9640 1 -- 172.21.15.7:0/4250845924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908804ac70 con 0x55aa9e32a160 2024-03-23T13:19:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f9084ff9640 1 -- 172.21.15.7:0/4250845924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908804a330 con 0x55aa9e32a160 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f90983b6740 1 -- 172.21.15.7:0/4250845924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 msgr2=0x55aa9e33abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f90983b6740 1 --2- 172.21.15.7:0/4250845924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33abb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9088009230 tx=0x7f9088005fd0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f90983b6740 1 -- 172.21.15.7:0/4250845924 shutdown_connections 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f90983b6740 1 --2- 172.21.15.7:0/4250845924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e32f2d0 0x55aa9e333830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f90983b6740 1 --2- 172.21.15.7:0/4250845924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33abb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f90983b6740 1 --2- 172.21.15.7:0/4250845924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e329770 0x55aa9e3332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f90983b6740 1 -- 172.21.15.7:0/4250845924 >> 172.21.15.7:0/4250845924 conn(0x55aa9e324ce0 msgr2=0x55aa9e32b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.158+0000 7f90983b6740 1 -- 172.21.15.7:0/4250845924 shutdown_connections 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f90983b6740 1 -- 172.21.15.7:0/4250845924 wait complete. 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f90983b6740 1 -- start start 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f90983b6740 1 Processor -- start 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f90983b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e329770 0x55aa9e33a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f90983b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f90983b6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32f2d0 0x55aa9e336ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f90983b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aa9e3374a0 con 0x55aa9e32a160 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f90983b6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aa9e337610 con 0x55aa9e32f2d0 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f90983b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aa9e3377d0 con 0x55aa9e329770 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f90983b6740 10 client.0 _ll_register_callbacks cb 0x7f90800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f908effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f908effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32784/0 (socket says 172.21.15.7:32784) 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f908effd640 1 -- 172.21.15.7:0/1441785939 learned_addr learned my addr 172.21.15.7:0/1441785939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f908e7fc640 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e329770 0x55aa9e33a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f908e7fc640 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32f2d0 0x55aa9e336ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f908effd640 1 -- 172.21.15.7:0/1441785939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e329770 msgr2=0x55aa9e33a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f908effd640 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e329770 0x55aa9e33a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f908effd640 1 -- 172.21.15.7:0/1441785939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32f2d0 msgr2=0x55aa9e336ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f908effd640 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32f2d0 0x55aa9e336ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.159+0000 7f908effd640 1 -- 172.21.15.7:0/1441785939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9088030070 con 0x55aa9e32a160 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f908effd640 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33a980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9088036040 tx=0x7f908804ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6686fb0ab5bab0e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90767fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90767fc640 1 -- 172.21.15.7:0/1441785939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90880046c0 con 0x55aa9e32a160 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90767fc640 1 -- 172.21.15.7:0/1441785939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9088046040 con 0x55aa9e32a160 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90983b6740 1 -- 172.21.15.7:0/1441785939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9078005500 con 0x55aa9e32a160 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90767fc640 1 -- 172.21.15.7:0/1441785939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9088004820 con 0x55aa9e32a160 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90983b6740 10 client.15273 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90983b6740 1 -- 172.21.15.7:0/1441785939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa9e24f7e0 con 0x55aa9e32a160 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90767fc640 1 -- 172.21.15.7:0/1441785939 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9078005500 con 0x55aa9e32a160 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90767fc640 1 -- 172.21.15.7:0/1441785939 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f908804a050 con 0x55aa9e32a160 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90983b6740 10 client.15273 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90983b6740 10 client.15273 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90983b6740 20 client.15273 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90983b6740 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55aa9e3aaac0 0x55aa9e3a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90983b6740 4 client.15273 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f90983b6740 1 -- 172.21.15.7:0/1441785939 --> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] -- command(tid 0: {"prefix": "status"}) -- 0x55aa9dfcb610 con 0x55aa9e3aaac0 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.160+0000 7f908e7fc640 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55aa9e3aaac0 0x55aa9e3a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f908e7fc640 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55aa9e3aaac0 0x55aa9e3a97d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a87b20a835502292 server_cookie=3ccbf0426ee63c62 in_seq=0 out_seq=0 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90767fc640 10 client.15273 ms_handle_connect on v2:172.21.15.155:6834/1880391142 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90767fc640 1 -- 172.21.15.7:0/1441785939 <== mds.4 v2:172.21.15.155:6834/1880391142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55aa9dfcb610 con 0x55aa9e3aaac0 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90767fc640 10 client.15273 handle_command_reply: tid=0 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 client.15273 shutdown 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 -- 172.21.15.7:0/1441785939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 msgr2=0x55aa9e33a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33a980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9088036040 tx=0x7f908804ac00 comp rx=0 tx=0).stop 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 -- 172.21.15.7:0/1441785939 shutdown_connections 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa9e32f2d0 0x55aa9e336ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.155:6834/1880391142,v1:172.21.15.155:6835/1880391142] conn(0x55aa9e3aaac0 0x55aa9e3a97d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa9e32a160 0x55aa9e33a980 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90767fc640 0 client.15273 ms_handle_reset on v2:172.21.15.155:6834/1880391142 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 --2- 172.21.15.7:0/1441785939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa9e329770 0x55aa9e33a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 -- 172.21.15.7:0/1441785939 >> 172.21.15.7:0/1441785939 conn(0x55aa9e324ce0 msgr2=0x55aa9e3332f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 -- 172.21.15.7:0/1441785939 shutdown_connections 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 1 -- 172.21.15.7:0/1441785939 wait complete. 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.161+0000 7f90983b6740 20 client.15273 trim_cache size 0 max 16384 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 4, 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24463, 2024-03-23T13:19:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-23T13:19:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-23T13:19:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-23T13:19:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 106.58917309900001, 2024-03-23T13:19:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-03-23T13:19:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-23T13:19:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T13:19:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 134.24319506000001 2024-03-23T13:19:34.170 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 -- 172.21.15.7:0/354345762 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f907c0a20c0 msgr2=0x7f907c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f907c0a20c0 0x7f907c0a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9080097380 tx=0x7f9080090400 comp rx=0 tx=0).stop 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 -- 172.21.15.7:0/354345762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9090102260 msgr2=0x7f909019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9090102260 0x7f909019dca0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f908000eb10 tx=0x7f908000c2d0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 -- 172.21.15.7:0/354345762 shutdown_connections 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f907c0a20c0 0x7f907c0a4580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9090102260 0x7f909019dca0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90901018a0 0x7f9090196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 --2- 172.21.15.7:0/354345762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90901006b0 0x7f90901966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 -- 172.21.15.7:0/354345762 >> 172.21.15.7:0/354345762 conn(0x7f90900fbc20 msgr2=0x7f9090104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 -- 172.21.15.7:0/354345762 shutdown_connections 2024-03-23T13:19:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.162+0000 7f90957dd640 1 -- 172.21.15.7:0/354345762 wait complete. 2024-03-23T13:19:34.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-23T13:19:34.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:33 smithi130 ceph-mon[36225]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:19:34.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:33 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/233257893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:34.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:33 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1898924519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:34.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:33 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3949795092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.276+0000 7fbd54464640 1 -- 172.21.15.7:0/827852549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd4c100620 msgr2=0x7fbd4c100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.276+0000 7fbd54464640 1 --2- 172.21.15.7:0/827852549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd4c100620 0x7fbd4c100a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009900 tx=0x7fbd4002bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.276+0000 7fbd54464640 1 -- 172.21.15.7:0/827852549 shutdown_connections 2024-03-23T13:19:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.276+0000 7fbd54464640 1 --2- 172.21.15.7:0/827852549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd4c1021d0 0x7fbd4c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.276+0000 7fbd54464640 1 --2- 172.21.15.7:0/827852549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd4c101810 0x7fbd4c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.276+0000 7fbd54464640 1 --2- 172.21.15.7:0/827852549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd4c100620 0x7fbd4c100a40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.276+0000 7fbd54464640 1 -- 172.21.15.7:0/827852549 >> 172.21.15.7:0/827852549 conn(0x7fbd4c0fbbb0 msgr2=0x7fbd4c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.276+0000 7fbd54464640 1 -- 172.21.15.7:0/827852549 shutdown_connections 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.276+0000 7fbd54464640 1 -- 172.21.15.7:0/827852549 wait complete. 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd54464640 1 Processor -- start 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd54464640 1 -- start start 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd54464640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd4c100620 0x7fbd4c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd54464640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd4c101810 0x7fbd4c197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd54464640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd4c1021d0 0x7fbd4c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd54464640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c19f2b0 con 0x7fbd4c100620 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd54464640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c19f420 con 0x7fbd4c101810 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd54464640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c19f5e0 con 0x7fbd4c1021d0 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd52200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd4c100620 0x7fbd4c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd52200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd4c100620 0x7fbd4c1977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32816/0 (socket says 172.21.15.7:32816) 2024-03-23T13:19:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd52200640 1 -- 172.21.15.7:0/2204365183 learned_addr learned my addr 172.21.15.7:0/2204365183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd519ff640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd4c101810 0x7fbd4c197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd52a01640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd4c1021d0 0x7fbd4c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd52200640 1 -- 172.21.15.7:0/2204365183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd4c1021d0 msgr2=0x7fbd4c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd52200640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd4c1021d0 0x7fbd4c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.277+0000 7fbd52200640 1 -- 172.21.15.7:0/2204365183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd4c101810 msgr2=0x7fbd4c197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.278+0000 7fbd52200640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd4c101810 0x7fbd4c197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.278+0000 7fbd52200640 1 -- 172.21.15.7:0/2204365183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd40009560 con 0x7fbd4c100620 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.278+0000 7fbd52200640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd4c100620 0x7fbd4c1977c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbd40005ab0 tx=0x7fbd40004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.278+0000 7fbd3f7fe640 1 -- 172.21.15.7:0/2204365183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40004aa0 con 0x7fbd4c100620 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.278+0000 7fbd54464640 1 -- 172.21.15.7:0/2204365183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c19f860 con 0x7fbd4c100620 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.278+0000 7fbd3f7fe640 1 -- 172.21.15.7:0/2204365183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd400052d0 con 0x7fbd4c100620 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.278+0000 7fbd3f7fe640 1 -- 172.21.15.7:0/2204365183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40003e80 con 0x7fbd4c100620 2024-03-23T13:19:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.278+0000 7fbd54464640 1 -- 172.21.15.7:0/2204365183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c19fd70 con 0x7fbd4c100620 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.280+0000 7fbd3f7fe640 1 -- 172.21.15.7:0/2204365183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fbd40005440 con 0x7fbd4c100620 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.280+0000 7fbd57044740 1 Processor -- start 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd57044740 1 -- start start 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd57044740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89c560 0x55629f89c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd57044740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89cf50 0x55629f8a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd57044740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8a4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd57044740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55629f80b180 con 0x55629f8a20c0 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd57044740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55629f8086b0 con 0x55629f89cf50 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd57044740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55629f803e90 con 0x55629f89c560 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd3f7fe640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd380a20c0 0x7fbd380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd519ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd52a01640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89c560 0x55629f89c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd52a01640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89c560 0x55629f89c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:47610/0 (socket says 172.21.15.7:47610) 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd52a01640 1 -- 172.21.15.7:0/2325734765 learned_addr learned my addr 172.21.15.7:0/2325734765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd519ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8a4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32830/0 (socket says 172.21.15.7:32830) 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd52200640 1 --2- 172.21.15.7:0/2325734765 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89cf50 0x55629f8a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd52a01640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd380a20c0 0x7fbd380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd3f7fe640 1 -- 172.21.15.7:0/2204365183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fbd400f0140 con 0x7fbd4c100620 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd52a01640 1 -- 172.21.15.7:0/2325734765 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89cf50 msgr2=0x55629f8a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd52a01640 1 --2- 172.21.15.7:0/2325734765 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89cf50 0x55629f8a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd52a01640 1 -- 172.21.15.7:0/2325734765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 msgr2=0x55629f8a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd52a01640 1 --2- 172.21.15.7:0/2325734765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.281+0000 7fbd52a01640 1 -- 172.21.15.7:0/2325734765 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629f7c0d80 con 0x55629f89c560 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.282+0000 7fbd52a01640 1 --2- 172.21.15.7:0/2325734765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89c560 0x55629f89c980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbd44012d80 tx=0x7fbd44010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a02941ad6a1dd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.282+0000 7fbd52a01640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd380a20c0 0x7fbd380a4580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009fd0 tx=0x7fbd4400c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.282+0000 7fbd3cbfe640 1 -- 172.21.15.7:0/2325734765 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440125a0 con 0x55629f89c560 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.282+0000 7fbd3cbfe640 1 -- 172.21.15.7:0/2325734765 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd44009d50 con 0x55629f89c560 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.282+0000 7fbd3cbfe640 1 -- 172.21.15.7:0/2325734765 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44003d80 con 0x55629f89c560 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 -- 172.21.15.7:0/2325734765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89c560 msgr2=0x55629f89c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 --2- 172.21.15.7:0/2325734765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89c560 0x55629f89c980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbd44012d80 tx=0x7fbd44010740 comp rx=0 tx=0).stop 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 -- 172.21.15.7:0/2325734765 shutdown_connections 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 --2- 172.21.15.7:0/2325734765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 --2- 172.21.15.7:0/2325734765 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89cf50 0x55629f8a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 --2- 172.21.15.7:0/2325734765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89c560 0x55629f89c980 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 -- 172.21.15.7:0/2325734765 >> 172.21.15.7:0/2325734765 conn(0x55629f897ad0 msgr2=0x55629f897ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 -- 172.21.15.7:0/2325734765 shutdown_connections 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 -- 172.21.15.7:0/2325734765 wait complete. 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 -- start start 2024-03-23T13:19:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 Processor -- start 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89c560 0x55629f8a6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89cf50 0x55629f8a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f8a20c0 0x55629f8adae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55629f7a8d50 con 0x55629f89c560 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55629f4d7310 con 0x55629f89cf50 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55629f7898f0 con 0x55629f8a20c0 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd57044740 10 client.0 _ll_register_callbacks cb 0x55629f514ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd52200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89c560 0x55629f8a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd52200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89c560 0x55629f8a6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32840/0 (socket says 172.21.15.7:32840) 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd52200640 1 -- 172.21.15.7:0/292823023 learned_addr learned my addr 172.21.15.7:0/292823023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd52a01640 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f8a20c0 0x55629f8adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.283+0000 7fbd519ff640 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89cf50 0x55629f8a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd52200640 1 -- 172.21.15.7:0/292823023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f8a20c0 msgr2=0x55629f8adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd52200640 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f8a20c0 0x55629f8adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd52200640 1 -- 172.21.15.7:0/292823023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89cf50 msgr2=0x55629f8a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd52200640 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89cf50 0x55629f8a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd52200640 1 -- 172.21.15.7:0/292823023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4401a040 con 0x55629f89c560 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd52200640 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89c560 0x55629f8a6560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbd400056f0 tx=0x7fbd40005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aed1309794a47d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd35ffb640 1 -- 172.21.15.7:0/292823023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd400bce20 con 0x55629f89c560 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd35ffb640 1 -- 172.21.15.7:0/292823023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fbd400c0f70 con 0x55629f89c560 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd57044740 1 -- 172.21.15.7:0/292823023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55629f5036c0 con 0x55629f89c560 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd35ffb640 1 -- 172.21.15.7:0/292823023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbd400c55c0 con 0x55629f89c560 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd57044740 10 client.15285 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd57044740 1 -- 172.21.15.7:0/292823023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55629f802510 con 0x55629f89c560 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd35ffb640 1 -- 172.21.15.7:0/292823023 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55629f5036c0 con 0x55629f89c560 2024-03-23T13:19:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd35ffb640 1 -- 172.21.15.7:0/292823023 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbd4003c070 con 0x55629f89c560 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd57044740 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.284+0000 7fbd57044740 10 client.15285 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.285+0000 7fbd57044740 20 client.15285 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.285+0000 7fbd57044740 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x55629f8b1c40 0x55629f8b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.285+0000 7fbd57044740 4 client.15285 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.285+0000 7fbd57044740 1 -- 172.21.15.7:0/292823023 --> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55629f554270 con 0x55629f8b1c40 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.285+0000 7fbd519ff640 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x55629f8b1c40 0x55629f8b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.285+0000 7fbd519ff640 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x55629f8b1c40 0x55629f8b3e30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9789f517664e4647 server_cookie=b33ba819d646aa8e in_seq=0 out_seq=0 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.285+0000 7fbd35ffb640 10 client.15285 ms_handle_connect on v2:172.21.15.130:6836/877142007 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd35ffb640 1 -- 172.21.15.7:0/292823023 <== mds.3 v2:172.21.15.130:6836/877142007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55629f554270 con 0x55629f8b1c40 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd35ffb640 10 client.15285 handle_command_reply: tid=0 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 client.15285 shutdown 2024-03-23T13:19:34.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 -- 172.21.15.7:0/292823023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89c560 msgr2=0x55629f8a6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89c560 0x55629f8a6560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbd400056f0 tx=0x7fbd40005740 comp rx=0 tx=0).stop 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 -- 172.21.15.7:0/292823023 shutdown_connections 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x55629f8b1c40 0x55629f8b3e30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f8a20c0 0x55629f8adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89cf50 0x55629f8a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd35ffb640 0 client.15285 ms_handle_reset on v2:172.21.15.130:6836/877142007 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 --2- 172.21.15.7:0/292823023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89c560 0x55629f8a6560 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 -- 172.21.15.7:0/292823023 >> 172.21.15.7:0/292823023 conn(0x55629f897ad0 msgr2=0x55629f896ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 -- 172.21.15.7:0/292823023 shutdown_connections 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 1 -- 172.21.15.7:0/292823023 wait complete. 2024-03-23T13:19:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.286+0000 7fbd57044740 20 client.15285 trim_cache size 0 max 16384 2024-03-23T13:19:34.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:33 smithi155 ceph-mon[36157]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T13:19:34.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:33 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/233257893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:19:34.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:33 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1898924519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:19:34.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:33 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3949795092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.304+0000 7fbd57044740 1 Processor -- start 2024-03-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.304+0000 7fbd57044740 1 -- start start 2024-03-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd57044740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89c560 0x55629f897eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd57044740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89cf50 0x55629f8ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd57044740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f8a20c0 0x55629f8a6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd57044740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55629f8b2020 con 0x55629f89cf50 2024-03-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd57044740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55629f89d640 con 0x55629f89c560 2024-03-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd57044740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55629f5542a0 con 0x55629f8a20c0 2024-03-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd519ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89cf50 0x55629f8ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd519ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89cf50 0x55629f8ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32850/0 (socket says 172.21.15.7:32850) 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd519ff640 1 -- 172.21.15.7:0/753340208 learned_addr learned my addr 172.21.15.7:0/753340208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd52a01640 1 --2- 172.21.15.7:0/753340208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f8a20c0 0x55629f8a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd52200640 1 --2- 172.21.15.7:0/753340208 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89c560 0x55629f897eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd519ff640 1 -- 172.21.15.7:0/753340208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f8a20c0 msgr2=0x55629f8a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd519ff640 1 --2- 172.21.15.7:0/753340208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f8a20c0 0x55629f8a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd519ff640 1 -- 172.21.15.7:0/753340208 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89c560 msgr2=0x55629f897eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd519ff640 1 --2- 172.21.15.7:0/753340208 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89c560 0x55629f897eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd519ff640 1 -- 172.21.15.7:0/753340208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629f79ea80 con 0x55629f89cf50 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd519ff640 1 --2- 172.21.15.7:0/753340208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89cf50 0x55629f8ada70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55629f8ae0a0 tx=0x7fbd48005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e85fd1a834172c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd37fff640 1 -- 172.21.15.7:0/753340208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48017030 con 0x55629f89cf50 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd37fff640 1 -- 172.21.15.7:0/753340208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd48018040 con 0x55629f89cf50 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.305+0000 7fbd37fff640 1 -- 172.21.15.7:0/753340208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48010a60 con 0x55629f89cf50 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 -- 172.21.15.7:0/753340208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89cf50 msgr2=0x55629f8ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 --2- 172.21.15.7:0/753340208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89cf50 0x55629f8ada70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55629f8ae0a0 tx=0x7fbd48005f90 comp rx=0 tx=0).stop 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 -- 172.21.15.7:0/753340208 shutdown_connections 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 --2- 172.21.15.7:0/753340208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f8a20c0 0x55629f8a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 --2- 172.21.15.7:0/753340208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f89cf50 0x55629f8ada70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 --2- 172.21.15.7:0/753340208 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89c560 0x55629f897eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 -- 172.21.15.7:0/753340208 >> 172.21.15.7:0/753340208 conn(0x55629f897ad0 msgr2=0x55629f892810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 -- 172.21.15.7:0/753340208 shutdown_connections 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 -- 172.21.15.7:0/753340208 wait complete. 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 -- start start 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.306+0000 7fbd57044740 1 Processor -- start 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd57044740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89c560 0x55629f8a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd57044740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89cf50 0x55629f8a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd57044740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd57044740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55629f8acff0 con 0x55629f8a20c0 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd57044740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55629f8ad160 con 0x55629f89c560 2024-03-23T13:19:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd57044740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55629f8ad320 con 0x55629f89cf50 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd57044740 10 client.0 _ll_register_callbacks cb 0x7fbd40032ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd52a01640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd52a01640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32860/0 (socket says 172.21.15.7:32860) 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd519ff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89cf50 0x55629f8a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd52a01640 1 -- 172.21.15.7:0/2837894019 learned_addr learned my addr 172.21.15.7:0/2837894019 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd52200640 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89c560 0x55629f8a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd52a01640 1 -- 172.21.15.7:0/2837894019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89cf50 msgr2=0x55629f8a9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd52a01640 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89cf50 0x55629f8a9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd52a01640 1 -- 172.21.15.7:0/2837894019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89c560 msgr2=0x55629f8a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd52a01640 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89c560 0x55629f8a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd52a01640 1 -- 172.21.15.7:0/2837894019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd48009b00 con 0x55629f8a20c0 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd519ff640 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89cf50 0x55629f8a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd52a01640 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8acab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbd44006010 tx=0x7fbd44005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec1a22419f40ada server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd35ffb640 1 -- 172.21.15.7:0/2837894019 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44003970 con 0x55629f8a20c0 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd35ffb640 1 -- 172.21.15.7:0/2837894019 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fbd44003ad0 con 0x55629f8a20c0 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.307+0000 7fbd57044740 1 -- 172.21.15.7:0/2837894019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd44009d50 con 0x55629f8a20c0 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd35ffb640 1 -- 172.21.15.7:0/2837894019 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbd44003c90 con 0x55629f8a20c0 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd57044740 10 client.15291 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd57044740 1 -- 172.21.15.7:0/2837894019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55629f7c47e0 con 0x55629f8a20c0 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd35ffb640 1 -- 172.21.15.7:0/2837894019 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd44009d50 con 0x55629f8a20c0 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd35ffb640 1 -- 172.21.15.7:0/2837894019 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbd44026020 con 0x55629f8a20c0 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd57044740 10 client.15291 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd57044740 10 client.15291 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd57044740 20 client.15291 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd57044740 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x55629f91d370 0x55629f91c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd57044740 4 client.15291 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T13:19:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd57044740 1 -- 172.21.15.7:0/2837894019 --> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] -- command(tid 0: {"prefix": "status"}) -- 0x55629f540610 con 0x55629f91d370 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd52200640 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x55629f91d370 0x55629f91c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd35ffb640 10 client.15291 ms_handle_connect on v2:172.21.15.130:6836/877142007 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.308+0000 7fbd52200640 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x55629f91d370 0x55629f91c080 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2670185cf8df4c9c server_cookie=eeb420512c9e9166 in_seq=0 out_seq=0 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd35ffb640 1 -- 172.21.15.7:0/2837894019 <== mds.3 v2:172.21.15.130:6836/877142007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55629f540610 con 0x55629f91d370 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd35ffb640 10 client.15291 handle_command_reply: tid=0 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 client.15291 shutdown 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 -- 172.21.15.7:0/2837894019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 msgr2=0x55629f8acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8acab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbd44006010 tx=0x7fbd44005c60 comp rx=0 tx=0).stop 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 -- 172.21.15.7:0/2837894019 shutdown_connections 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55629f8a20c0 0x55629f8acab0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.130:6836/877142007,v1:172.21.15.130:6837/877142007] conn(0x55629f91d370 0x55629f91c080 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55629f89cf50 0x55629f8a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 --2- 172.21.15.7:0/2837894019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55629f89c560 0x55629f8a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 -- 172.21.15.7:0/2837894019 >> 172.21.15.7:0/2837894019 conn(0x55629f897ad0 msgr2=0x55629f892810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd35ffb640 0 client.15291 ms_handle_reset on v2:172.21.15.130:6836/877142007 2024-03-23T13:19:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 -- 172.21.15.7:0/2837894019 shutdown_connections 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 1 -- 172.21.15.7:0/2837894019 wait complete. 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd57044740 20 client.15291 trim_cache size 0 max 16384 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 3, 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24470, 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 107.743693166, 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-03-23T13:19:34.317 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-23T13:19:34.318 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T13:19:34.318 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 128.69784353399999 2024-03-23T13:19:34.318 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:19:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd54464640 1 -- 172.21.15.7:0/2204365183 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd380a20c0 msgr2=0x7fbd380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd54464640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd380a20c0 0x7fbd380a4580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009fd0 tx=0x7fbd4400c000 comp rx=0 tx=0).stop 2024-03-23T13:19:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd54464640 1 -- 172.21.15.7:0/2204365183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd4c100620 msgr2=0x7fbd4c1977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.309+0000 7fbd54464640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd4c100620 0x7fbd4c1977c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbd40005ab0 tx=0x7fbd40004270 comp rx=0 tx=0).stop 2024-03-23T13:19:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.310+0000 7fbd54464640 1 -- 172.21.15.7:0/2204365183 shutdown_connections 2024-03-23T13:19:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.310+0000 7fbd54464640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbd380a20c0 0x7fbd380a4580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.310+0000 7fbd54464640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd4c1021d0 0x7fbd4c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.310+0000 7fbd54464640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd4c101810 0x7fbd4c197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.310+0000 7fbd54464640 1 --2- 172.21.15.7:0/2204365183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd4c100620 0x7fbd4c1977c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.310+0000 7fbd54464640 1 -- 172.21.15.7:0/2204365183 >> 172.21.15.7:0/2204365183 conn(0x7fbd4c0fbbb0 msgr2=0x7fbd4c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.310+0000 7fbd54464640 1 -- 172.21.15.7:0/2204365183 shutdown_connections 2024-03-23T13:19:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.310+0000 7fbd54464640 1 -- 172.21.15.7:0/2204365183 wait complete. 2024-03-23T13:19:34.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T13:19:34.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.424+0000 7f3038dd0640 1 -- 172.21.15.7:0/2014846968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30340ffa00 msgr2=0x7f30340ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.424+0000 7f3038dd0640 1 --2- 172.21.15.7:0/2014846968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30340ffa00 0x7f30340ffe20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3024009ca0 tx=0x7f302402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.425+0000 7f3038dd0640 1 -- 172.21.15.7:0/2014846968 shutdown_connections 2024-03-23T13:19:34.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.425+0000 7f3038dd0640 1 --2- 172.21.15.7:0/2014846968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3034103390 0x7f3034109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.425+0000 7f3038dd0640 1 --2- 172.21.15.7:0/2014846968 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30341029d0 0x7f3034102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.425+0000 7f3038dd0640 1 --2- 172.21.15.7:0/2014846968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30340ffa00 0x7f30340ffe20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.425+0000 7f3038dd0640 1 -- 172.21.15.7:0/2014846968 >> 172.21.15.7:0/2014846968 conn(0x7f30340fb330 msgr2=0x7f30340fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.425+0000 7f3038dd0640 1 -- 172.21.15.7:0/2014846968 shutdown_connections 2024-03-23T13:19:34.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.425+0000 7f3038dd0640 1 -- 172.21.15.7:0/2014846968 wait complete. 2024-03-23T13:19:34.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.425+0000 7f3038dd0640 1 Processor -- start 2024-03-23T13:19:34.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.425+0000 7f3038dd0640 1 -- start start 2024-03-23T13:19:34.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3038dd0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30340ffa00 0x7f3034197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3038dd0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30341029d0 0x7f3034197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3038dd0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3034103390 0x7f303419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3038dd0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f303419f320 con 0x7f3034103390 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3038dd0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f303419f490 con 0x7f30341029d0 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3038dd0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f303419f650 con 0x7f30340ffa00 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3032d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3034103390 0x7f303419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3032d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3034103390 0x7f303419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32872/0 (socket says 172.21.15.7:32872) 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3032d9d640 1 -- 172.21.15.7:0/2352022823 learned_addr learned my addr 172.21.15.7:0/2352022823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f303259c640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30340ffa00 0x7f3034197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30341029d0 0x7f3034197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3032d9d640 1 -- 172.21.15.7:0/2352022823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30340ffa00 msgr2=0x7f3034197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3032d9d640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30340ffa00 0x7f3034197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3032d9d640 1 -- 172.21.15.7:0/2352022823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30341029d0 msgr2=0x7f3034197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3032d9d640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30341029d0 0x7f3034197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3032d9d640 1 -- 172.21.15.7:0/2352022823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3024009900 con 0x7f3034103390 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3032d9d640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3034103390 0x7f303419ede0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f302c00e970 tx=0x7f302c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f30237fe640 1 -- 172.21.15.7:0/2352022823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c009720 con 0x7f3034103390 2024-03-23T13:19:34.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f30237fe640 1 -- 172.21.15.7:0/2352022823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c005500 con 0x7f3034103390 2024-03-23T13:19:34.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f30237fe640 1 -- 172.21.15.7:0/2352022823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c010430 con 0x7f3034103390 2024-03-23T13:19:34.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3038dd0640 1 -- 172.21.15.7:0/2352022823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303419f930 con 0x7f3034103390 2024-03-23T13:19:34.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.426+0000 7f3038dd0640 1 -- 172.21.15.7:0/2352022823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303419fe90 con 0x7f3034103390 2024-03-23T13:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.428+0000 7f30237fe640 1 -- 172.21.15.7:0/2352022823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f302c004ff0 con 0x7f3034103390 2024-03-23T13:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303b9b4740 1 Processor -- start 2024-03-23T13:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303b9b4740 1 -- start start 2024-03-23T13:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b519792560 0x55b519792980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b519797a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b5197980c0 0x55b51979a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303b9b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b519701180 con 0x55b519792f50 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303b9b4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b5196fe6b0 con 0x55b5197980c0 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303b9b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5196f9e90 con 0x55b519792560 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f30237fe640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f301c0a2140 0x7f301c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b519797a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b519797a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32882/0 (socket says 172.21.15.7:32882) 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f303259c640 1 -- 172.21.15.7:0/2702303796 learned_addr learned my addr 172.21.15.7:0/2702303796 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.429+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2702303796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b519792560 0x55b519792980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f3032d9d640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f301c0a2140 0x7f301c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2702303796 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b5197980c0 0x55b51979a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f30237fe640 1 -- 172.21.15.7:0/2352022823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f302c014070 con 0x7f3034103390 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f303259c640 1 -- 172.21.15.7:0/2702303796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b519792560 msgr2=0x55b519792980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f303259c640 1 --2- 172.21.15.7:0/2702303796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b519792560 0x55b519792980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f303259c640 1 -- 172.21.15.7:0/2702303796 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b5197980c0 msgr2=0x55b51979a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f303259c640 1 --2- 172.21.15.7:0/2702303796 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b5197980c0 0x55b51979a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f303259c640 1 -- 172.21.15.7:0/2702303796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5196b6d80 con 0x55b519792f50 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2702303796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b519792560 0x55b519792980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f303259c640 1 --2- 172.21.15.7:0/2702303796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b519797a50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3034069a80 tx=0x7f30240093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02b4d657521bfad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f3032d9d640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f301c0a2140 0x7f301c0a4600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f302c095730 tx=0x7f302c06dd50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f3020ff9640 1 -- 172.21.15.7:0/2702303796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30240323c0 con 0x55b519792f50 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f3020ff9640 1 -- 172.21.15.7:0/2702303796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3024034440 con 0x55b519792f50 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.430+0000 7f3020ff9640 1 -- 172.21.15.7:0/2702303796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024003b90 con 0x55b519792f50 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 -- 172.21.15.7:0/2702303796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 msgr2=0x55b519797a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2702303796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b519797a50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3034069a80 tx=0x7f30240093f0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 -- 172.21.15.7:0/2702303796 shutdown_connections 2024-03-23T13:19:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2702303796 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b5197980c0 0x55b51979a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2702303796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b519797a50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2702303796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b519792560 0x55b519792980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 -- 172.21.15.7:0/2702303796 >> 172.21.15.7:0/2702303796 conn(0x55b51978dad0 msgr2=0x55b51978ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 -- 172.21.15.7:0/2702303796 shutdown_connections 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 -- 172.21.15.7:0/2702303796 wait complete. 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 -- start start 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.431+0000 7f303b9b4740 1 Processor -- start 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 0x55b51979c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b51979ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b5197a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303b9b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b51969ed50 con 0x55b519792f50 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303b9b4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b5193cd310 con 0x55b519792560 2024-03-23T13:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303b9b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b51967f8f0 con 0x55b5197980c0 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303b9b4740 10 client.0 _ll_register_callbacks cb 0x55b51940aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b51979ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b51979ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32890/0 (socket says 172.21.15.7:32890) 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303259c640 1 -- 172.21.15.7:0/2531078734 learned_addr learned my addr 172.21.15.7:0/2531078734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 0x55b51979c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b5197a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303259c640 1 -- 172.21.15.7:0/2531078734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 msgr2=0x55b5197a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303259c640 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b5197a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303259c640 1 -- 172.21.15.7:0/2531078734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 msgr2=0x55b51979c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303259c640 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 0x55b51979c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303259c640 1 -- 172.21.15.7:0/2531078734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303410a460 con 0x55b519792f50 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303259c640 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b51979ca20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3024037040 tx=0x7f3024034ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7147f0bf8eff524c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f3019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f3019ffb640 1 -- 172.21.15.7:0/2531078734 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30240329b0 con 0x55b519792f50 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f3019ffb640 1 -- 172.21.15.7:0/2531078734 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3024003970 con 0x55b519792f50 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303b9b4740 1 -- 172.21.15.7:0/2531078734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5193f96c0 con 0x55b519792f50 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f3019ffb640 1 -- 172.21.15.7:0/2531078734 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3024032b10 con 0x55b519792f50 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303b9b4740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.432+0000 7f303b9b4740 1 -- 172.21.15.7:0/2531078734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b5196f8510 con 0x55b519792f50 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f3019ffb640 1 -- 172.21.15.7:0/2531078734 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b5193f96c0 con 0x55b519792f50 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f3019ffb640 1 -- 172.21.15.7:0/2531078734 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f302404a450 con 0x55b519792f50 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f303b9b4740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f303b9b4740 10 client.15303 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f303b9b4740 20 client.15303 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55b5197a7bc0 0x55b5197a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f303b9b4740 4 client.15303 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f303b9b4740 1 -- 172.21.15.7:0/2531078734 --> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b51944a270 con 0x55b5197a7bc0 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55b5197a7bc0 0x55b5197a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55b5197a7bc0 0x55b5197a9d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76c9d50a9e5db6a0 server_cookie=767a460e51336d77 in_seq=0 out_seq=0 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.433+0000 7f3019ffb640 10 client.15303 ms_handle_connect on v2:172.21.15.155:6836/3999767587 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f3019ffb640 1 -- 172.21.15.7:0/2531078734 <== mds.2 v2:172.21.15.155:6836/3999767587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b51944a270 con 0x55b5197a7bc0 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f3019ffb640 10 client.15303 handle_command_reply: tid=0 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 client.15303 shutdown 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 -- 172.21.15.7:0/2531078734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 msgr2=0x55b51979ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b51979ca20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3024037040 tx=0x7f3024034ca0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 -- 172.21.15.7:0/2531078734 shutdown_connections 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55b5197a7bc0 0x55b5197a9d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b5197a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f3019ffb640 0 client.15303 ms_handle_reset on v2:172.21.15.155:6836/3999767587 2024-03-23T13:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b51979ca20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2531078734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 0x55b51979c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 -- 172.21.15.7:0/2531078734 >> 172.21.15.7:0/2531078734 conn(0x55b51978dad0 msgr2=0x55b51978cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 -- 172.21.15.7:0/2531078734 shutdown_connections 2024-03-23T13:19:34.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 1 -- 172.21.15.7:0/2531078734 wait complete. 2024-03-23T13:19:34.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.434+0000 7f303b9b4740 20 client.15303 trim_cache size 0 max 16384 2024-03-23T13:19:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f303b9b4740 1 Processor -- start 2024-03-23T13:19:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f303b9b4740 1 -- start start 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792560 0x55b51978deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792f50 0x55b5197a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b51978c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f303b9b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b51978d990 con 0x55b519792560 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f303b9b4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b519797f80 con 0x55b519792f50 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f303b9b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b51979a6e0 con 0x55b5197980c0 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f3031d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792560 0x55b51978deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f3031d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792560 0x55b51978deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32906/0 (socket says 172.21.15.7:32906) 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f3031d9b640 1 -- 172.21.15.7:0/2128269669 learned_addr learned my addr 172.21.15.7:0/2128269669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.452+0000 7f303259c640 1 --2- 172.21.15.7:0/2128269669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792f50 0x55b5197a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2128269669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b51978c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f3031d9b640 1 -- 172.21.15.7:0/2128269669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 msgr2=0x55b51978c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2128269669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b51978c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f3031d9b640 1 -- 172.21.15.7:0/2128269669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792f50 msgr2=0x55b5197a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2128269669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792f50 0x55b5197a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f3031d9b640 1 -- 172.21.15.7:0/2128269669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b519694a80 con 0x55b519792560 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f3031d9b640 1 --2- 172.21.15.7:0/2128269669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792560 0x55b51978deb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3034198d40 tx=0x7f30280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716c073f10f111cc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f301bfff640 1 -- 172.21.15.7:0/2128269669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028016040 con 0x55b519792560 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f301bfff640 1 -- 172.21.15.7:0/2128269669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3028012810 con 0x55b519792560 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f301bfff640 1 -- 172.21.15.7:0/2128269669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028004a00 con 0x55b519792560 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.453+0000 7f303b9b4740 1 -- 172.21.15.7:0/2128269669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792560 msgr2=0x55b51978deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2128269669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792560 0x55b51978deb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3034198d40 tx=0x7f30280046c0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 -- 172.21.15.7:0/2128269669 shutdown_connections 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2128269669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b51978c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2128269669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792f50 0x55b5197a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 --2- 172.21.15.7:0/2128269669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792560 0x55b51978deb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 -- 172.21.15.7:0/2128269669 >> 172.21.15.7:0/2128269669 conn(0x55b51978dad0 msgr2=0x55b519797870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 -- 172.21.15.7:0/2128269669 shutdown_connections 2024-03-23T13:19:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 -- 172.21.15.7:0/2128269669 wait complete. 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 -- start start 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 Processor -- start 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 0x55b51979fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b5197a0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b5197a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b5197a4320 con 0x55b519792f50 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b5197a8450 con 0x55b519792560 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5197a3130 con 0x55b5197980c0 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303b9b4740 10 client.0 _ll_register_callbacks cb 0x7f3028009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b5197a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b5197a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32922/0 (socket says 172.21.15.7:32922) 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303259c640 1 -- 172.21.15.7:0/1398018677 learned_addr learned my addr 172.21.15.7:0/1398018677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f3031d9b640 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 0x55b51979fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f3031d9b640 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b5197a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303259c640 1 -- 172.21.15.7:0/1398018677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 msgr2=0x55b5197a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303259c640 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b5197a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303259c640 1 -- 172.21.15.7:0/1398018677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 msgr2=0x55b51979fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303259c640 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 0x55b51979fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f303259c640 1 -- 172.21.15.7:0/1398018677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30280044b0 con 0x55b519792f50 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.454+0000 7f3031d9b640 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 0x55b51979fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f303259c640 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b5197a0190 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f302404a800 tx=0x7f3024009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c212eee9c9c3c98 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f3019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f3019ffb640 1 -- 172.21.15.7:0/1398018677 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30240323c0 con 0x55b519792f50 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f3019ffb640 1 -- 172.21.15.7:0/1398018677 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f30240327b0 con 0x55b519792f50 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f303b9b4740 1 -- 172.21.15.7:0/1398018677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3024034440 con 0x55b519792f50 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f3019ffb640 1 -- 172.21.15.7:0/1398018677 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f30240329b0 con 0x55b519792f50 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f303b9b4740 10 client.15309 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f303b9b4740 1 -- 172.21.15.7:0/1398018677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b5196ba7e0 con 0x55b519792f50 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f3019ffb640 1 -- 172.21.15.7:0/1398018677 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302406e020 con 0x55b519792f50 2024-03-23T13:19:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f3019ffb640 1 -- 172.21.15.7:0/1398018677 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3024004040 con 0x55b519792f50 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f303b9b4740 10 client.15309 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f303b9b4740 10 client.15309 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f303b9b4740 20 client.15309 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f303b9b4740 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55b5197a7bc0 0x55b5198119a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f303b9b4740 4 client.15309 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.455+0000 7f303b9b4740 1 -- 172.21.15.7:0/1398018677 --> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] -- command(tid 0: {"prefix": "status"}) -- 0x55b519436610 con 0x55b5197a7bc0 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f3031d9b640 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55b5197a7bc0 0x55b5198119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f3019ffb640 10 client.15309 ms_handle_connect on v2:172.21.15.155:6836/3999767587 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f3031d9b640 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55b5197a7bc0 0x55b5198119a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58027391b1da7f14 server_cookie=e46587bc79d26e67 in_seq=0 out_seq=0 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f3019ffb640 1 -- 172.21.15.7:0/1398018677 <== mds.2 v2:172.21.15.155:6836/3999767587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55b519436610 con 0x55b5197a7bc0 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f3019ffb640 10 client.15309 handle_command_reply: tid=0 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 client.15309 shutdown 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 -- 172.21.15.7:0/1398018677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 msgr2=0x55b5197a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b5197a0190 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f302404a800 tx=0x7f3024009540 comp rx=0 tx=0).stop 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 -- 172.21.15.7:0/1398018677 shutdown_connections 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.155:6836/3999767587,v1:172.21.15.155:6837/3999767587] conn(0x55b5197a7bc0 0x55b5198119a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5197980c0 0x55b5197a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f3019ffb640 0 client.15309 ms_handle_reset on v2:172.21.15.155:6836/3999767587 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b519792f50 0x55b5197a0190 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 --2- 172.21.15.7:0/1398018677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b519792560 0x55b51979fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 -- 172.21.15.7:0/1398018677 >> 172.21.15.7:0/1398018677 conn(0x55b51978dad0 msgr2=0x55b519794270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 -- 172.21.15.7:0/1398018677 shutdown_connections 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 1 -- 172.21.15.7:0/1398018677 wait complete. 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.456+0000 7f303b9b4740 20 client.15309 trim_cache size 0 max 16384 2024-03-23T13:19:34.464 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24475, 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 108.904328738, 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 131.691904962 2024-03-23T13:19:34.465 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:19:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 -- 172.21.15.7:0/2352022823 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f301c0a2140 msgr2=0x7f301c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f301c0a2140 0x7f301c0a4600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f302c095730 tx=0x7f302c06dd50 comp rx=0 tx=0).stop 2024-03-23T13:19:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 -- 172.21.15.7:0/2352022823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3034103390 msgr2=0x7f303419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3034103390 0x7f303419ede0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f302c00e970 tx=0x7f302c00ed70 comp rx=0 tx=0).stop 2024-03-23T13:19:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 -- 172.21.15.7:0/2352022823 shutdown_connections 2024-03-23T13:19:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f301c0a2140 0x7f301c0a4600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3034103390 0x7f303419ede0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30341029d0 0x7f3034197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 --2- 172.21.15.7:0/2352022823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30340ffa00 0x7f3034197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 -- 172.21.15.7:0/2352022823 >> 172.21.15.7:0/2352022823 conn(0x7f30340fb330 msgr2=0x7f3034105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 -- 172.21.15.7:0/2352022823 shutdown_connections 2024-03-23T13:19:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.457+0000 7f3038dd0640 1 -- 172.21.15.7:0/2352022823 wait complete. 2024-03-23T13:19:34.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-23T13:19:34.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 -- 172.21.15.7:0/3431721988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e74101240 msgr2=0x7f8e74108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 --2- 172.21.15.7:0/3431721988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e74101240 0x7f8e74108ae0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009ca0 tx=0x7f8e6c02c140 comp rx=0 tx=0).stop 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 -- 172.21.15.7:0/3431721988 shutdown_connections 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 --2- 172.21.15.7:0/3431721988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e74101240 0x7f8e74108ae0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 --2- 172.21.15.7:0/3431721988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e74100880 0x7f8e74100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 --2- 172.21.15.7:0/3431721988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e74101cc0 0x7f8e741020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 -- 172.21.15.7:0/3431721988 >> 172.21.15.7:0/3431721988 conn(0x7f8e740fb310 msgr2=0x7f8e740fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 -- 172.21.15.7:0/3431721988 shutdown_connections 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 -- 172.21.15.7:0/3431721988 wait complete. 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 Processor -- start 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 -- start start 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e74100880 0x7f8e74100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.572+0000 7f8e7aa15640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e74101240 0x7f8e7419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e7aa15640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e74101cc0 0x7f8e7419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e7aa15640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7419f280 con 0x7f8e74101240 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e7aa15640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7419f3f0 con 0x7f8e74100880 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e7aa15640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7419f5b0 con 0x7f8e74101cc0 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e74101240 0x7f8e7419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e74101240 0x7f8e7419c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42046/0 (socket says 172.21.15.7:42046) 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e737fe640 1 -- 172.21.15.7:0/2109448255 learned_addr learned my addr 172.21.15.7:0/2109448255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e73fff640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e74100880 0x7f8e74100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e74101cc0 0x7f8e7419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e737fe640 1 -- 172.21.15.7:0/2109448255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e74101cc0 msgr2=0x7f8e7419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e737fe640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e74101cc0 0x7f8e7419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e737fe640 1 -- 172.21.15.7:0/2109448255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e74100880 msgr2=0x7f8e74100300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e737fe640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e74100880 0x7f8e74100300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e737fe640 1 -- 172.21.15.7:0/2109448255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c009900 con 0x7f8e74101240 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e737fe640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e74101240 0x7f8e7419c4b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e9c0 tx=0x7f8e6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e717fa640 1 -- 172.21.15.7:0/2109448255 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68009700 con 0x7f8e74101240 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e717fa640 1 -- 172.21.15.7:0/2109448255 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6800cd80 con 0x7f8e74101240 2024-03-23T13:19:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e7aa15640 1 -- 172.21.15.7:0/2109448255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7419f890 con 0x7f8e74101240 2024-03-23T13:19:34.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e717fa640 1 -- 172.21.15.7:0/2109448255 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68010430 con 0x7f8e74101240 2024-03-23T13:19:34.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.573+0000 7f8e7aa15640 1 -- 172.21.15.7:0/2109448255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7419fda0 con 0x7f8e74101240 2024-03-23T13:19:34.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e717fa640 1 -- 172.21.15.7:0/2109448255 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f8e68003680 con 0x7f8e74101240 2024-03-23T13:19:34.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e7d5f3740 1 Processor -- start 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e7d5f3740 1 -- start start 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126137980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x55912613ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x55912613f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5591260a6180 con 0x55912613d0c0 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5591260a36b0 con 0x559126137560 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55912609ee90 con 0x559126137f50 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e717fa640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8e5c0a2140 0x7f8e5c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x55912613f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e78fb2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126137980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e73fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x55912613ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e78fb2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126137980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:36130/0 (socket says 172.21.15.7:36130) 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e78fb2640 1 -- 172.21.15.7:0/3185469013 learned_addr learned my addr 172.21.15.7:0/3185469013 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.576+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8e5c0a2140 0x7f8e5c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e717fa640 1 -- 172.21.15.7:0/2109448255 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8e68014070 con 0x7f8e74101240 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e73fff640 1 -- 172.21.15.7:0/3185469013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 msgr2=0x559126137980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e73fff640 1 --2- 172.21.15.7:0/3185469013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126137980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e73fff640 1 -- 172.21.15.7:0/3185469013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 msgr2=0x55912613f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e73fff640 1 --2- 172.21.15.7:0/3185469013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x55912613f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e73fff640 1 -- 172.21.15.7:0/3185469013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55912605bd80 con 0x559126137f50 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/3185469013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8e5c0a2140 0x7f8e5c0a4600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5591261387b0 tx=0x7f8e6c0326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e73fff640 1 --2- 172.21.15.7:0/3185469013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x55912613ca50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009c00 tx=0x7f8e64011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17a267406fc8340a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e5a7fc640 1 -- 172.21.15.7:0/3185469013 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64004230 con 0x559126137f50 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e5a7fc640 1 -- 172.21.15.7:0/3185469013 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e64009e70 con 0x559126137f50 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.577+0000 7f8e5a7fc640 1 -- 172.21.15.7:0/3185469013 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64004cd0 con 0x559126137f50 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.578+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3185469013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 msgr2=0x55912613ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.578+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3185469013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x55912613ca50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009c00 tx=0x7f8e64011810 comp rx=0 tx=0).stop 2024-03-23T13:19:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.578+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3185469013 shutdown_connections 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.578+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3185469013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x55912613f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.578+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3185469013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x55912613ca50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.578+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3185469013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.578+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3185469013 >> 172.21.15.7:0/3185469013 conn(0x559126132ad0 msgr2=0x559126132ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.578+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3185469013 shutdown_connections 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.578+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3185469013 wait complete. 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.578+0000 7f8e7d5f3740 1 -- start start 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e7d5f3740 1 Processor -- start 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559126137560 0x559126141540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55912613d0c0 0x559126148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559126043d50 con 0x559126137560 2024-03-23T13:19:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559125d72310 con 0x55912613d0c0 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5591260248f0 con 0x559126137f50 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e7d5f3740 10 client.0 _ll_register_callbacks cb 0x559125dafac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e73fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559126137560 0x559126141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e73fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559126137560 0x559126141540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42072/0 (socket says 172.21.15.7:42072) 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e73fff640 1 -- 172.21.15.7:0/3432754647 learned_addr learned my addr 172.21.15.7:0/3432754647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e737fe640 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55912613d0c0 0x559126148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e73fff640 1 -- 172.21.15.7:0/3432754647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 msgr2=0x559126141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e73fff640 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e73fff640 1 -- 172.21.15.7:0/3432754647 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55912613d0c0 msgr2=0x559126148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e73fff640 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55912613d0c0 0x559126148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e73fff640 1 -- 172.21.15.7:0/3432754647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e64009830 con 0x559126137560 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e73fff640 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559126137560 0x559126141540 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005890 tx=0x7f8e64019a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf80cf59e55a635a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e537fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e537fe640 1 -- 172.21.15.7:0/3432754647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64019e70 con 0x559126137560 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.579+0000 7f8e537fe640 1 -- 172.21.15.7:0/3432754647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8e64024470 con 0x559126137560 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3432754647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559125d9e6c0 con 0x559126137560 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e537fe640 1 -- 172.21.15.7:0/3432754647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e64024630 con 0x559126137560 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e7d5f3740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3432754647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55912609d510 con 0x559126137560 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e537fe640 1 -- 172.21.15.7:0/3432754647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559125d9e6c0 con 0x559126137560 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e537fe640 1 -- 172.21.15.7:0/3432754647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8e640239a0 con 0x559126137560 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e7d5f3740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e7d5f3740 10 client.15321 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e7d5f3740 20 client.15321 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55912614dcc0 0x55912614f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e7d5f3740 4 client.15321 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3432754647 --> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559125def270 con 0x55912614dcc0 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.580+0000 7f8e737fe640 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55912614dcc0 0x55912614f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e737fe640 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55912614dcc0 0x55912614f130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=caf12b6d12f2b52a server_cookie=2bab814562ba9eaf in_seq=0 out_seq=0 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e537fe640 10 client.15321 ms_handle_connect on v2:172.21.15.155:6838/131946068 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e537fe640 1 -- 172.21.15.7:0/3432754647 <== mds.1 v2:172.21.15.155:6838/131946068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559125def270 con 0x55912614dcc0 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e537fe640 10 client.15321 handle_command_reply: tid=0 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e7d5f3740 1 client.15321 shutdown 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3432754647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559126137560 msgr2=0x559126141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559126137560 0x559126141540 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005890 tx=0x7f8e64019a80 comp rx=0 tx=0).stop 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3432754647 shutdown_connections 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x55912614dcc0 0x55912614f130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55912613d0c0 0x559126148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3432754647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559126137560 0x559126141540 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e537fe640 0 client.15321 ms_handle_reset on v2:172.21.15.155:6838/131946068 2024-03-23T13:19:34.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.581+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3432754647 >> 172.21.15.7:0/3432754647 conn(0x559126132ad0 msgr2=0x559126131ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.582+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3432754647 shutdown_connections 2024-03-23T13:19:34.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.582+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3432754647 wait complete. 2024-03-23T13:19:34.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.582+0000 7f8e7d5f3740 20 client.15321 trim_cache size 0 max 16384 2024-03-23T13:19:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e7d5f3740 1 Processor -- start 2024-03-23T13:19:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e7d5f3740 1 -- start start 2024-03-23T13:19:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126132eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126140b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x5591261410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559126149380 con 0x55912613d0c0 2024-03-23T13:19:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559126137120 con 0x559126137560 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55912614e0a0 con 0x559126137f50 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e78fb2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x5591261410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e78fb2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x5591261410a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42074/0 (socket says 172.21.15.7:42074) 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e78fb2640 1 -- 172.21.15.7:0/3760832299 learned_addr learned my addr 172.21.15.7:0/3760832299 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e73fff640 1 --2- 172.21.15.7:0/3760832299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126132eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e737fe640 1 --2- 172.21.15.7:0/3760832299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e78fb2640 1 -- 172.21.15.7:0/3760832299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 msgr2=0x559126140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/3760832299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e78fb2640 1 -- 172.21.15.7:0/3760832299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 msgr2=0x559126132eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/3760832299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126132eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.600+0000 7f8e78fb2640 1 -- 172.21.15.7:0/3760832299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559126039a80 con 0x55912613d0c0 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/3760832299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x5591261410a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0325e0 tx=0x7f8e6c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f7bb3f9e6df70a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e59ffb640 1 -- 172.21.15.7:0/3760832299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c032a70 con 0x55912613d0c0 2024-03-23T13:19:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e59ffb640 1 -- 172.21.15.7:0/3760832299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c003970 con 0x55912613d0c0 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e59ffb640 1 -- 172.21.15.7:0/3760832299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c004530 con 0x55912613d0c0 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3760832299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 msgr2=0x5591261410a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3760832299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x5591261410a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0325e0 tx=0x7f8e6c03c000 comp rx=0 tx=0).stop 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3760832299 shutdown_connections 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3760832299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x5591261410a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3760832299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/3760832299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126132eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3760832299 >> 172.21.15.7:0/3760832299 conn(0x559126132ad0 msgr2=0x55912614ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3760832299 shutdown_connections 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.601+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/3760832299 wait complete. 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e7d5f3740 1 -- start start 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e7d5f3740 1 Processor -- start 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126144690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e7d5f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x559126147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559126148130 con 0x55912613d0c0 2024-03-23T13:19:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5591261482a0 con 0x559126137560 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e7d5f3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559126148460 con 0x559126137f50 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e7d5f3740 10 client.0 _ll_register_callbacks cb 0x7f8e680046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e78fb2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x559126147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e78fb2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x559126147bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42086/0 (socket says 172.21.15.7:42086) 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e78fb2640 1 -- 172.21.15.7:0/4214522484 learned_addr learned my addr 172.21.15.7:0/4214522484 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e73fff640 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e737fe640 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e78fb2640 1 -- 172.21.15.7:0/4214522484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 msgr2=0x559126144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e78fb2640 1 -- 172.21.15.7:0/4214522484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 msgr2=0x559126144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.602+0000 7f8e78fb2640 1 -- 172.21.15.7:0/4214522484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c030070 con 0x55912613d0c0 2024-03-23T13:19:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e78fb2640 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x559126147bf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c02c680 tx=0x7f8e6c0323e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874e281a9b1f7d7e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e537fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e537fe640 1 -- 172.21.15.7:0/4214522484 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c003ae0 con 0x55912613d0c0 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e537fe640 1 -- 172.21.15.7:0/4214522484 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8e6c05e030 con 0x55912613d0c0 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/4214522484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e64009e70 con 0x55912613d0c0 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e537fe640 1 -- 172.21.15.7:0/4214522484 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e6c003c40 con 0x55912613d0c0 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e7d5f3740 10 client.15327 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/4214522484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55912605f7e0 con 0x55912613d0c0 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e537fe640 1 -- 172.21.15.7:0/4214522484 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e64009e70 con 0x55912613d0c0 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e537fe640 1 -- 172.21.15.7:0/4214522484 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8e6c058650 con 0x55912613d0c0 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e7d5f3740 10 client.15327 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e7d5f3740 10 client.15327 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e7d5f3740 20 client.15327 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x5591261b8860 0x5591261b7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e7d5f3740 4 client.15327 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.603+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/4214522484 --> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] -- command(tid 0: {"prefix": "status"}) -- 0x559125ddb610 con 0x5591261b8860 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e73fff640 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x5591261b8860 0x5591261b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e73fff640 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x5591261b8860 0x5591261b7570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82ea7d5555ed0b66 server_cookie=d6439e3709854d2b in_seq=0 out_seq=0 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e537fe640 10 client.15327 ms_handle_connect on v2:172.21.15.155:6838/131946068 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e537fe640 1 -- 172.21.15.7:0/4214522484 <== mds.1 v2:172.21.15.155:6838/131946068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x559125ddb610 con 0x5591261b8860 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e537fe640 10 client.15327 handle_command_reply: tid=0 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 client.15327 shutdown 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/4214522484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 msgr2=0x559126147bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x559126147bf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c02c680 tx=0x7f8e6c0323e0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/4214522484 shutdown_connections 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.155:6838/131946068,v1:172.21.15.155:6839/131946068] conn(0x5591261b8860 0x5591261b7570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55912613d0c0 0x559126147bf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559126137f50 0x559126144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 --2- 172.21.15.7:0/4214522484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559126137560 0x559126144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e537fe640 0 client.15327 ms_handle_reset on v2:172.21.15.155:6838/131946068 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/4214522484 >> 172.21.15.7:0/4214522484 conn(0x559126132ad0 msgr2=0x559126132eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/4214522484 shutdown_connections 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 1 -- 172.21.15.7:0/4214522484 wait complete. 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.604+0000 7f8e7d5f3740 20 client.15327 trim_cache size 0 max 16384 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24496, 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-23T13:19:34.612 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 110.058332209, 2024-03-23T13:19:34.613 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-03-23T13:19:34.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-23T13:19:34.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T13:19:34.613 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 126.27850382 2024-03-23T13:19:34.613 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:19:34.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 -- 172.21.15.7:0/2109448255 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8e5c0a2140 msgr2=0x7f8e5c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8e5c0a2140 0x7f8e5c0a4600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5591261387b0 tx=0x7f8e6c0326b0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 -- 172.21.15.7:0/2109448255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e74101240 msgr2=0x7f8e7419c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e74101240 0x7f8e7419c4b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e9c0 tx=0x7f8e6800ee80 comp rx=0 tx=0).stop 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 -- 172.21.15.7:0/2109448255 shutdown_connections 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f8e5c0a2140 0x7f8e5c0a4600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e74101cc0 0x7f8e7419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e74101240 0x7f8e7419c4b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 --2- 172.21.15.7:0/2109448255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e74100880 0x7f8e74100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 -- 172.21.15.7:0/2109448255 >> 172.21.15.7:0/2109448255 conn(0x7f8e740fb310 msgr2=0x7f8e74104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 -- 172.21.15.7:0/2109448255 shutdown_connections 2024-03-23T13:19:34.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.605+0000 7f8e7aa15640 1 -- 172.21.15.7:0/2109448255 wait complete. 2024-03-23T13:19:34.617 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T13:19:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.719+0000 7f26ce9f6640 1 -- 172.21.15.7:0/3795322783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8101820 msgr2=0x7f26c8101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.719+0000 7f26ce9f6640 1 --2- 172.21.15.7:0/3795322783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8101820 0x7f26c8101ca0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009900 tx=0x7f26bc02bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.719+0000 7f26ce9f6640 1 -- 172.21.15.7:0/3795322783 shutdown_connections 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.719+0000 7f26ce9f6640 1 --2- 172.21.15.7:0/3795322783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c81021e0 0x7f26c8108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.719+0000 7f26ce9f6640 1 --2- 172.21.15.7:0/3795322783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8101820 0x7f26c8101ca0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.719+0000 7f26ce9f6640 1 --2- 172.21.15.7:0/3795322783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c8100630 0x7f26c8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.719+0000 7f26ce9f6640 1 -- 172.21.15.7:0/3795322783 >> 172.21.15.7:0/3795322783 conn(0x7f26c80fbba0 msgr2=0x7f26c80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.719+0000 7f26ce9f6640 1 -- 172.21.15.7:0/3795322783 shutdown_connections 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.719+0000 7f26ce9f6640 1 -- 172.21.15.7:0/3795322783 wait complete. 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26ce9f6640 1 Processor -- start 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26ce9f6640 1 -- start start 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26ce9f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c8100630 0x7f26c806a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26ce9f6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8101820 0x7f26c819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26ce9f6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c81021e0 0x7f26c819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26ce9f6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26c819f320 con 0x7f26c8100630 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26ce9f6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f26c819f490 con 0x7f26c8101820 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26ce9f6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26c819f650 con 0x7f26c81021e0 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c8100630 0x7f26c806a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c8100630 0x7f26c806a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42114/0 (socket says 172.21.15.7:42114) 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26c7fff640 1 -- 172.21.15.7:0/559299005 learned_addr learned my addr 172.21.15.7:0/559299005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26ccf93640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c81021e0 0x7f26c819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.720+0000 7f26c77fe640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8101820 0x7f26c819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26c7fff640 1 -- 172.21.15.7:0/559299005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c81021e0 msgr2=0x7f26c819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26c7fff640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c81021e0 0x7f26c819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26c7fff640 1 -- 172.21.15.7:0/559299005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8101820 msgr2=0x7f26c819c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26c7fff640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8101820 0x7f26c819c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26c7fff640 1 -- 172.21.15.7:0/559299005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc009560 con 0x7f26c8100630 2024-03-23T13:19:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26c7fff640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c8100630 0x7f26c806a4f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e9c0 tx=0x7f26b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26c57fa640 1 -- 172.21.15.7:0/559299005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80096e0 con 0x7f26c8100630 2024-03-23T13:19:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26c57fa640 1 -- 172.21.15.7:0/559299005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b800cd80 con 0x7f26c8100630 2024-03-23T13:19:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26c57fa640 1 -- 172.21.15.7:0/559299005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8010430 con 0x7f26c8100630 2024-03-23T13:19:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26ce9f6640 1 -- 172.21.15.7:0/559299005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c819f930 con 0x7f26c8100630 2024-03-23T13:19:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.721+0000 7f26ce9f6640 1 -- 172.21.15.7:0/559299005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c819fdc0 con 0x7f26c8100630 2024-03-23T13:19:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.723+0000 7f26c57fa640 1 -- 172.21.15.7:0/559299005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f26b8003680 con 0x7f26c8100630 2024-03-23T13:19:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.723+0000 7f26d15cc740 1 Processor -- start 2024-03-23T13:19:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.723+0000 7f26d15cc740 1 -- start start 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c57fa640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26b00a20c0 0x7f26b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878732d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 0x558787332a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587873330c0 0x558787335550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26d15cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55878729c180 con 0x55878732d560 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26d15cc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5587872996b0 con 0x5587873330c0 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26d15cc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558787294e90 con 0x55878732df50 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26ccf93640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26b00a20c0 0x7f26b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878732d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878732d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42124/0 (socket says 172.21.15.7:42124) 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c7fff640 1 -- 172.21.15.7:0/407233869 learned_addr learned my addr 172.21.15.7:0/407233869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26ccf93640 1 --2- 172.21.15.7:0/407233869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587873330c0 0x558787335550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c77fe640 1 --2- 172.21.15.7:0/407233869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 0x558787332a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c57fa640 1 -- 172.21.15.7:0/559299005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f26b8014070 con 0x7f26c8100630 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c7fff640 1 -- 172.21.15.7:0/407233869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 msgr2=0x558787332a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c7fff640 1 --2- 172.21.15.7:0/407233869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 0x558787332a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c7fff640 1 -- 172.21.15.7:0/407233869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587873330c0 msgr2=0x558787335550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c7fff640 1 --2- 172.21.15.7:0/407233869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587873330c0 0x558787335550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c7fff640 1 -- 172.21.15.7:0/407233869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558787251d80 con 0x55878732d560 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26ccf93640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26b00a20c0 0x7f26b00a4580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005fd0 tx=0x7f26c0009b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26c7fff640 1 --2- 172.21.15.7:0/407233869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878732d980 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f26b8097730 tx=0x7f26b8007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a611644ee75de1b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26ae7fc640 1 -- 172.21.15.7:0/407233869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b805f0e0 con 0x55878732d560 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26ae7fc640 1 -- 172.21.15.7:0/407233869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b809a8a0 con 0x55878732d560 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.724+0000 7f26ae7fc640 1 -- 172.21.15.7:0/407233869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b809ad60 con 0x55878732d560 2024-03-23T13:19:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.725+0000 7f26d15cc740 1 -- 172.21.15.7:0/407233869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 msgr2=0x55878732d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.725+0000 7f26d15cc740 1 --2- 172.21.15.7:0/407233869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878732d980 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f26b8097730 tx=0x7f26b8007850 comp rx=0 tx=0).stop 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.725+0000 7f26d15cc740 1 -- 172.21.15.7:0/407233869 shutdown_connections 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.725+0000 7f26d15cc740 1 --2- 172.21.15.7:0/407233869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587873330c0 0x558787335550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.725+0000 7f26d15cc740 1 --2- 172.21.15.7:0/407233869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 0x558787332a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.725+0000 7f26d15cc740 1 --2- 172.21.15.7:0/407233869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878732d980 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.725+0000 7f26d15cc740 1 -- 172.21.15.7:0/407233869 >> 172.21.15.7:0/407233869 conn(0x558787328ad0 msgr2=0x558787328ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.725+0000 7f26d15cc740 1 -- 172.21.15.7:0/407233869 shutdown_connections 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.725+0000 7f26d15cc740 1 -- 172.21.15.7:0/407233869 wait complete. 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.725+0000 7f26d15cc740 1 -- start start 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26d15cc740 1 Processor -- start 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732d560 0x558787337520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732df50 0x558787337a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587873330c0 0x55878733eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26d15cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558787239d50 con 0x55878732df50 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26d15cc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558786f68310 con 0x5587873330c0 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26d15cc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55878721a8f0 con 0x55878732d560 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26d15cc740 10 client.0 _ll_register_callbacks cb 0x558786fa5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26c77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732df50 0x558787337a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26c77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732df50 0x558787337a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42132/0 (socket says 172.21.15.7:42132) 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26c77fe640 1 -- 172.21.15.7:0/2755789129 learned_addr learned my addr 172.21.15.7:0/2755789129 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26c7fff640 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732d560 0x558787337520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26ccf93640 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587873330c0 0x55878733eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26c77fe640 1 -- 172.21.15.7:0/2755789129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732d560 msgr2=0x558787337520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26c77fe640 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732d560 0x558787337520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26c77fe640 1 -- 172.21.15.7:0/2755789129 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587873330c0 msgr2=0x55878733eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26c77fe640 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587873330c0 0x55878733eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26c77fe640 1 -- 172.21.15.7:0/2755789129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c819fdc0 con 0x55878732df50 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26c77fe640 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732df50 0x558787337a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009900 tx=0x7f26bc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fc8d2cdf408af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.726+0000 7f26a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26a77fe640 1 -- 172.21.15.7:0/2755789129 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc004b20 con 0x55878732df50 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26a77fe640 1 -- 172.21.15.7:0/2755789129 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f26bc004250 con 0x55878732df50 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26d15cc740 1 -- 172.21.15.7:0/2755789129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558786f946c0 con 0x55878732df50 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26a77fe640 1 -- 172.21.15.7:0/2755789129 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26bc050030 con 0x55878732df50 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26d15cc740 10 client.15339 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26d15cc740 1 -- 172.21.15.7:0/2755789129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558787293510 con 0x55878732df50 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26a77fe640 1 -- 172.21.15.7:0/2755789129 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558786f946c0 con 0x55878732df50 2024-03-23T13:19:34.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26a77fe640 1 -- 172.21.15.7:0/2755789129 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f26bc03c040 con 0x55878732df50 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26d15cc740 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26d15cc740 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26d15cc740 20 client.15339 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558787343dd0 0x558787345240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26d15cc740 4 client.15339 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26d15cc740 1 -- 172.21.15.7:0/2755789129 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558786fe5270 con 0x558787343dd0 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.727+0000 7f26c7fff640 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558787343dd0 0x558787345240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.728+0000 7f26c7fff640 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558787343dd0 0x558787345240 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c751c7d0f6e7125b server_cookie=63828bc71144ead7 in_seq=0 out_seq=0 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.728+0000 7f26a77fe640 10 client.15339 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.728+0000 7f26a77fe640 1 -- 172.21.15.7:0/2755789129 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558786fe5270 con 0x558787343dd0 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.728+0000 7f26a77fe640 10 client.15339 handle_command_reply: tid=0 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.728+0000 7f26d15cc740 1 client.15339 shutdown 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 1 -- 172.21.15.7:0/2755789129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732df50 msgr2=0x558787337a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732df50 0x558787337a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009900 tx=0x7f26bc0048f0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 1 -- 172.21.15.7:0/2755789129 shutdown_connections 2024-03-23T13:19:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558787343dd0 0x558787345240 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587873330c0 0x55878733eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732df50 0x558787337a60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2755789129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732d560 0x558787337520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26a77fe640 0 client.15339 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 1 -- 172.21.15.7:0/2755789129 >> 172.21.15.7:0/2755789129 conn(0x558787328ad0 msgr2=0x558787327ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 1 -- 172.21.15.7:0/2755789129 shutdown_connections 2024-03-23T13:19:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 1 -- 172.21.15.7:0/2755789129 wait complete. 2024-03-23T13:19:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.729+0000 7f26d15cc740 20 client.15339 trim_cache size 0 max 16384 2024-03-23T13:19:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26d15cc740 1 Processor -- start 2024-03-23T13:19:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26d15cc740 1 -- start start 2024-03-23T13:19:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558787328ad0 0x558787336290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878733ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 0x558787342de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26d15cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55878733f360 con 0x55878732d560 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26d15cc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5587873441b0 con 0x558787328ad0 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26d15cc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558786f946c0 con 0x55878732df50 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26c77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878733ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26c77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878733ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42148/0 (socket says 172.21.15.7:42148) 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26c77fe640 1 -- 172.21.15.7:0/2498729874 learned_addr learned my addr 172.21.15.7:0/2498729874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26c7fff640 1 --2- 172.21.15.7:0/2498729874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558787328ad0 0x558787336290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.747+0000 7f26ccf93640 1 --2- 172.21.15.7:0/2498729874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 0x558787342de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26c77fe640 1 -- 172.21.15.7:0/2498729874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 msgr2=0x558787342de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26c77fe640 1 --2- 172.21.15.7:0/2498729874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 0x558787342de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26c77fe640 1 -- 172.21.15.7:0/2498729874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558787328ad0 msgr2=0x558787336290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26c77fe640 1 --2- 172.21.15.7:0/2498729874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558787328ad0 0x558787336290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26c77fe640 1 -- 172.21.15.7:0/2498729874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55878722fa80 con 0x55878732d560 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26ccf93640 1 --2- 172.21.15.7:0/2498729874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 0x558787342de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26c77fe640 1 --2- 172.21.15.7:0/2498729874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878733ea60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f26bc001eb0 tx=0x7f26bc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fb959170461870 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26adffb640 1 -- 172.21.15.7:0/2498729874 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc059070 con 0x55878732d560 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26adffb640 1 -- 172.21.15.7:0/2498729874 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26bc0035f0 con 0x55878732d560 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26adffb640 1 -- 172.21.15.7:0/2498729874 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc050070 con 0x55878732d560 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26d15cc740 1 -- 172.21.15.7:0/2498729874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 msgr2=0x55878733ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2498729874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878733ea60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f26bc001eb0 tx=0x7f26bc02bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26d15cc740 1 -- 172.21.15.7:0/2498729874 shutdown_connections 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2498729874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732df50 0x558787342de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2498729874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55878732d560 0x55878733ea60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.748+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2498729874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558787328ad0 0x558787336290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 -- 172.21.15.7:0/2498729874 >> 172.21.15.7:0/2498729874 conn(0x55878732eb50 msgr2=0x558787327ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 -- 172.21.15.7:0/2498729874 shutdown_connections 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 -- 172.21.15.7:0/2498729874 wait complete. 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 -- start start 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 Processor -- start 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558787328ad0 0x55878733c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732d560 0x5587873a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55878732df50 0x5587873a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558787332080 con 0x558787328ad0 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5587873a85e0 con 0x55878732df50 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5587873a87a0 con 0x55878732d560 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26d15cc740 10 client.0 _ll_register_callbacks cb 0x7f26bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558787328ad0 0x55878733c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558787328ad0 0x55878733c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42152/0 (socket says 172.21.15.7:42152) 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26c7fff640 1 -- 172.21.15.7:0/2187704139 learned_addr learned my addr 172.21.15.7:0/2187704139 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26c77fe640 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732d560 0x5587873a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.749+0000 7f26ccf93640 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55878732df50 0x5587873a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26c7fff640 1 -- 172.21.15.7:0/2187704139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732d560 msgr2=0x5587873a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26c7fff640 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732d560 0x5587873a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26c7fff640 1 -- 172.21.15.7:0/2187704139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55878732df50 msgr2=0x5587873a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26c7fff640 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55878732df50 0x5587873a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26c7fff640 1 -- 172.21.15.7:0/2187704139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558787293510 con 0x558787328ad0 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26c7fff640 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558787328ad0 0x55878733c9b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f26b809aa60 tx=0x7f26b8003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499f381e41bee4c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26a77fe640 1 -- 172.21.15.7:0/2187704139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b809b460 con 0x558787328ad0 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26a77fe640 1 -- 172.21.15.7:0/2187704139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f26b809ad60 con 0x558787328ad0 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26d15cc740 1 -- 172.21.15.7:0/2187704139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b809a8a0 con 0x558787328ad0 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26a77fe640 1 -- 172.21.15.7:0/2187704139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26b80b9250 con 0x558787328ad0 2024-03-23T13:19:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26d15cc740 10 client.15345 fetch_fsmap learned FSMap version 26 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26d15cc740 1 -- 172.21.15.7:0/2187704139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5587872557e0 con 0x558787328ad0 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.750+0000 7f26a77fe640 1 -- 172.21.15.7:0/2187704139 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b809a8a0 con 0x558787328ad0 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26a77fe640 1 -- 172.21.15.7:0/2187704139 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f26b801f060 con 0x558787328ad0 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26d15cc740 10 client.15345 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26d15cc740 10 client.15345 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26d15cc740 20 client.15345 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5587873330c0 0x5587873acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26d15cc740 4 client.15345 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26d15cc740 1 -- 172.21.15.7:0/2187704139 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "status"}) -- 0x558786fd1610 con 0x5587873330c0 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26c77fe640 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5587873330c0 0x5587873acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26c77fe640 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5587873330c0 0x5587873acbf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25314ebc7b8be170 server_cookie=17cf13a858eb4248 in_seq=0 out_seq=0 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26a77fe640 10 client.15345 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26a77fe640 1 -- 172.21.15.7:0/2187704139 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558786fd1610 con 0x5587873330c0 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26a77fe640 10 client.15345 handle_command_reply: tid=0 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.751+0000 7f26d15cc740 1 client.15345 shutdown 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 1 -- 172.21.15.7:0/2187704139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558787328ad0 msgr2=0x55878733c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558787328ad0 0x55878733c9b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f26b809aa60 tx=0x7f26b8003a20 comp rx=0 tx=0).stop 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 1 -- 172.21.15.7:0/2187704139 shutdown_connections 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5587873330c0 0x5587873acbf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55878732df50 0x5587873a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55878732d560 0x5587873a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26a77fe640 0 client.15345 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 1 --2- 172.21.15.7:0/2187704139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558787328ad0 0x55878733c9b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 1 -- 172.21.15.7:0/2187704139 >> 172.21.15.7:0/2187704139 conn(0x55878732eb50 msgr2=0x558787327ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 1 -- 172.21.15.7:0/2187704139 shutdown_connections 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 1 -- 172.21.15.7:0/2187704139 wait complete. 2024-03-23T13:19:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26d15cc740 20 client.15345 trim_cache size 0 max 16384 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "745f591a-e916-11ee-95cd-87774f69a715", 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24503, 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 113.230414568, 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T13:19:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 117.73423544000001 2024-03-23T13:19:34.761 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:19:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 -- 172.21.15.7:0/559299005 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26b00a20c0 msgr2=0x7f26b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26b00a20c0 0x7f26b00a4580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005fd0 tx=0x7f26c0009b80 comp rx=0 tx=0).stop 2024-03-23T13:19:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 -- 172.21.15.7:0/559299005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c8100630 msgr2=0x7f26c806a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c8100630 0x7f26c806a4f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e9c0 tx=0x7f26b800ee80 comp rx=0 tx=0).stop 2024-03-23T13:19:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 -- 172.21.15.7:0/559299005 shutdown_connections 2024-03-23T13:19:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f26b00a20c0 0x7f26b00a4580 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c81021e0 0x7f26c819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8101820 0x7f26c819c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 --2- 172.21.15.7:0/559299005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c8100630 0x7f26c806a4f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 -- 172.21.15.7:0/559299005 >> 172.21.15.7:0/559299005 conn(0x7f26c80fbba0 msgr2=0x7f26c8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 -- 172.21.15.7:0/559299005 shutdown_connections 2024-03-23T13:19:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:34.752+0000 7f26ce9f6640 1 -- 172.21.15.7:0/559299005 wait complete. 2024-03-23T13:19:34.764 INFO:teuthology.run_tasks:Running task sequential... 2024-03-23T13:19:34.774 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-23T13:19:34.774 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-23T13:19:34.774 INFO:teuthology.task.print:Enabling mgr modules 2024-03-23T13:19:34.774 INFO:teuthology.run_tasks:Running task kclient... 2024-03-23T13:19:34.832 INFO:tasks.kclient:Mounting kernel clients... 2024-03-23T13:19:34.832 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-23T13:19:34.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:19:34.832 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-23T13:19:34.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:19:34.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-03-23T13:19:34.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:34 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1493458197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:34.916 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-03-23T13:19:34.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:19:34.917 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-23T13:19:34.917 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T13:19:34.918 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T13:19:34.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-23T13:19:35.096 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:34 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1493458197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:35.198 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:19:35.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:34 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1493458197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:19:36.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:35 smithi007 ceph-mon[32184]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T13:19:36.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:35 smithi130 ceph-mon[36225]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T13:19:36.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:35 smithi155 ceph-mon[36157]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T13:19:36.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.893+0000 7fcc5e854700 1 -- 172.21.15.7:0/783881574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc58101100 msgr2=0x7fcc58101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.893+0000 7fcc5e854700 1 --2- 172.21.15.7:0/783881574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc58101100 0x7fcc58101560 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009990 tx=0x7fcc4c009ca0 comp rx=0 tx=0).stop 2024-03-23T13:19:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.893+0000 7fcc5e854700 1 -- 172.21.15.7:0/783881574 shutdown_connections 2024-03-23T13:19:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.893+0000 7fcc5e854700 1 --2- 172.21.15.7:0/783881574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc58101aa0 0x7fcc5810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.893+0000 7fcc5e854700 1 --2- 172.21.15.7:0/783881574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc58107140 0x7fcc58107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.893+0000 7fcc5e854700 1 --2- 172.21.15.7:0/783881574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc58101100 0x7fcc58101560 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.893+0000 7fcc5e854700 1 -- 172.21.15.7:0/783881574 >> 172.21.15.7:0/783881574 conn(0x7fcc58078030 msgr2=0x7fcc58078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.893+0000 7fcc5e854700 1 -- 172.21.15.7:0/783881574 shutdown_connections 2024-03-23T13:19:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.893+0000 7fcc5e854700 1 -- 172.21.15.7:0/783881574 wait complete. 2024-03-23T13:19:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5e854700 1 Processor -- start 2024-03-23T13:19:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5e854700 1 -- start start 2024-03-23T13:19:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5e854700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc58101100 0x7fcc58199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5e854700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc58101aa0 0x7fcc58199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5e854700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc58107140 0x7fcc581a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5e854700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5819a420 con 0x7fcc58107140 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5e854700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581a4c00 con 0x7fcc58101100 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5e854700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581a4d40 con 0x7fcc58101aa0 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5cdf1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc58107140 0x7fcc581a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5cdf1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc58107140 0x7fcc581a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42166/0 (socket says 172.21.15.7:42166) 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc5cdf1700 1 -- 172.21.15.7:0/1529671525 learned_addr learned my addr 172.21.15.7:0/1529671525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc577fe700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc58101aa0 0x7fcc58199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.894+0000 7fcc57fff700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc58101100 0x7fcc58199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc5cdf1700 1 -- 172.21.15.7:0/1529671525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc58101aa0 msgr2=0x7fcc58199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc5cdf1700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc58101aa0 0x7fcc58199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc5cdf1700 1 -- 172.21.15.7:0/1529671525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc58101100 msgr2=0x7fcc58199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc5cdf1700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc58101100 0x7fcc58199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:36.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc5cdf1700 1 -- 172.21.15.7:0/1529671525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c0095f0 con 0x7fcc58107140 2024-03-23T13:19:36.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc5cdf1700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc58107140 0x7fcc581a46c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000ef60 tx=0x7fcc5000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:36.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc557fa700 1 -- 172.21.15.7:0/1529671525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50018070 con 0x7fcc58107140 2024-03-23T13:19:36.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc557fa700 1 -- 172.21.15.7:0/1529671525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc5000f040 con 0x7fcc58107140 2024-03-23T13:19:36.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc5e854700 1 -- 172.21.15.7:0/1529671525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc581a4f90 con 0x7fcc58107140 2024-03-23T13:19:36.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc557fa700 1 -- 172.21.15.7:0/1529671525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000cd60 con 0x7fcc58107140 2024-03-23T13:19:36.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.895+0000 7fcc5e854700 1 -- 172.21.15.7:0/1529671525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc581a53b0 con 0x7fcc58107140 2024-03-23T13:19:36.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.896+0000 7fcc5e854700 1 -- 172.21.15.7:0/1529671525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc58109660 con 0x7fcc58107140 2024-03-23T13:19:36.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.897+0000 7fcc557fa700 1 -- 172.21.15.7:0/1529671525 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fcc50004ad0 con 0x7fcc58107140 2024-03-23T13:19:36.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.898+0000 7fcc557fa700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc400a2150 0x7fcc400a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:36.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.898+0000 7fcc57fff700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc400a2150 0x7fcc400a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:36.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.898+0000 7fcc557fa700 1 -- 172.21.15.7:0/1529671525 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcc500cd720 con 0x7fcc58107140 2024-03-23T13:19:36.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.898+0000 7fcc57fff700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc400a2150 0x7fcc400a4610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009be0 tx=0x7fcc48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:36.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:36.902+0000 7fcc557fa700 1 -- 172.21.15.7:0/1529671525 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fcc50096730 con 0x7fcc58107140 2024-03-23T13:19:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.150+0000 7fcc5e854700 1 -- 172.21.15.7:0/1529671525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcc58068ec0 con 0x7fcc58107140 2024-03-23T13:19:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.150+0000 7fcc557fa700 1 -- 172.21.15.7:0/1529671525 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fcc50099df0 con 0x7fcc58107140 2024-03-23T13:19:37.152 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.152+0000 7fcc5e854700 1 -- 172.21.15.7:0/1529671525 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc400a2150 msgr2=0x7fcc400a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.152+0000 7fcc5e854700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc400a2150 0x7fcc400a4610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009be0 tx=0x7fcc48009260 comp rx=0 tx=0).stop 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.152+0000 7fcc5e854700 1 -- 172.21.15.7:0/1529671525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc58107140 msgr2=0x7fcc581a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.152+0000 7fcc5e854700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc58107140 0x7fcc581a46c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000ef60 tx=0x7fcc5000ef90 comp rx=0 tx=0).stop 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.153+0000 7fcc5e854700 1 -- 172.21.15.7:0/1529671525 shutdown_connections 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.153+0000 7fcc5e854700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcc400a2150 0x7fcc400a4610 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.153+0000 7fcc5e854700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc58107140 0x7fcc581a46c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.153+0000 7fcc5e854700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc58101100 0x7fcc58199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.153+0000 7fcc5e854700 1 --2- 172.21.15.7:0/1529671525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc58101aa0 0x7fcc58199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.153+0000 7fcc5e854700 1 -- 172.21.15.7:0/1529671525 >> 172.21.15.7:0/1529671525 conn(0x7fcc58078030 msgr2=0x7fcc580ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.153+0000 7fcc5e854700 1 -- 172.21.15.7:0/1529671525 shutdown_connections 2024-03-23T13:19:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:37.153+0000 7fcc5e854700 1 -- 172.21.15.7:0/1529671525 wait complete. 2024-03-23T13:19:37.910 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T13:19:37.910 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T13:19:37.910 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-03-23T13:19:37.910 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T13:19:37.910 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T13:19:37.910 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T13:19:37.910 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T13:19:37.910 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T13:19:37.910 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T13:19:37.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:19:37.911 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-03-23T13:19:37.928 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T13:19:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T13:19:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T13:19:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-23T13:19:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-03-23T13:19:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-23T13:19:37.928 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T13:19:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42120sec preferred_lft 42120sec 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-03-23T13:19:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-03-23T13:19:37.930 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T13:19:37.930 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T13:19:37.930 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-23T13:19:37.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-03-23T13:19:37.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-03-23T13:19:37.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-03-23T13:19:37.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T13:19:37.930 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-23T13:19:38.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T13:19:38.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:37 smithi007 ceph-mon[32184]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-23T13:19:38.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:37 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1529671525' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T13:19:38.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T13:19:38.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:19:38.128 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T13:19:38.155 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-03-23T13:19:38.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:19:38.157 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-03-23T13:19:38.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:37 smithi130 ceph-mon[36225]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-23T13:19:38.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:37 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1529671525' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T13:19:38.232 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-03-23T13:19:38.232 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T13:19:38.232 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-23T13:19:38.232 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-23T13:19:38.232 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T13:19:38.233 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T13:19:38.233 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-23T13:19:38.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-23T13:19:38.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-23T13:19:38.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-23T13:19:38.233 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-23T13:19:38.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:37 smithi155 ceph-mon[36157]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-23T13:19:38.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:37 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1529671525' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T13:19:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T13:19:38.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T13:19:38.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:19:38.588 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-23T13:19:38.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:19:38.603 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-03-23T13:19:38.657 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T13:19:38.657 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-23T13:19:38.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T13:19:38.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-23T13:19:38.657 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-23T13:19:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T13:19:38.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T13:19:38.781 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-23T13:19:38.781 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T13:19:38.781 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-23T13:19:38.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-23T13:19:38.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T13:19:38.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-23T13:19:38.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-23T13:19:38.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-23T13:19:38.781 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-23T13:19:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T13:19:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T13:19:38.916 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T13:19:38.917 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-23T13:19:38.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-03-23T13:19:38.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T13:19:38.917 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-23T13:19:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T13:19:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T13:19:38.998 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T13:19:38.998 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T13:19:39.012 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T13:19:39.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T13:19:39.012 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T13:19:39.067 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-23T13:19:39.067 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-23T13:19:39.067 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/dd5ff27a-26e2-40d7-8415-18d36aa3eebc 2024-03-23T13:19:39.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:19:39.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/dd5ff27a-26e2-40d7-8415-18d36aa3eebc /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-03-23T13:19:39.150 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-03-23T13:19:39.150 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-03-23T13:19:39.150 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-03-23T13:19:39.278 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-03-23T13:19:39.279 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-03-23T13:19:39.279 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-03-23T13:19:39.279 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.130:3300,172.21.15.155:3300" 2024-03-23T13:19:39.279 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:3300,172.21.15.130:3300,172.21.15.155:3300:/volumes/qa/sv_1/dd5ff27a-26e2-40d7-8415-18d36aa3eebc 2024-03-23T13:19:39.279 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=745f591a-e916-11ee-95cd-87774f69a715" will pass to kernel 2024-03-23T13:19:39.280 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-23T13:19:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-03-23T13:19:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-23T13:19:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-03-23T13:19:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-23T13:19:39.281 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-23T13:19:39.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:19:39.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T13:19:39.353 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T13:19:39.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-23T13:19:39.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T13:19:39.450 INFO:teuthology.orchestra.run.smithi007.stdout:client25201 2024-03-23T13:19:39.450 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:19:39.450 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T13:19:39.466 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T13:19:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-23T13:19:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T13:19:39.562 INFO:teuthology.orchestra.run.smithi007.stdout:client25201 2024-03-23T13:19:39.562 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T13:19:39.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-23T13:19:39.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T13:19:39.618 INFO:teuthology.orchestra.run.smithi007.stdout:client25201 2024-03-23T13:19:39.618 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/745f591a-e916-11ee-95cd-87774f69a715.client25201/status') 2024-03-23T13:19:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T13:19:39.679 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.25201 (3)172.21.15.7:0/2881851906 2024-03-23T13:19:39.680 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-03-23T13:19:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-03-23T13:19:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-03-23T13:19:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:70 bytes copied, 2.1349e-05 s, 3.3 MB/s 2024-03-23T13:19:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T13:19:39.687 INFO:teuthology.run_tasks:Running task exec... 2024-03-23T13:19:39.697 INFO:teuthology.task.exec:Executing custom commands... 2024-03-23T13:19:39.697 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-23T13:19:39.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-23T13:19:39.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 -- 172.21.15.7:0/49600955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1940101840 msgr2=0x7f1940101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:39.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 --2- 172.21.15.7:0/49600955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1940101840 0x7f1940101cc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1928009900 tx=0x7f192802bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:39.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 -- 172.21.15.7:0/49600955 shutdown_connections 2024-03-23T13:19:39.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 --2- 172.21.15.7:0/49600955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1940102200 0x7f1940108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:39.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 --2- 172.21.15.7:0/49600955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1940101840 0x7f1940101cc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:39.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 --2- 172.21.15.7:0/49600955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1940100650 0x7f1940100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:39.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 -- 172.21.15.7:0/49600955 >> 172.21.15.7:0/49600955 conn(0x7f19400fbbc0 msgr2=0x7f19400fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:39.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 -- 172.21.15.7:0/49600955 shutdown_connections 2024-03-23T13:19:39.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 -- 172.21.15.7:0/49600955 wait complete. 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 Processor -- start 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 -- start start 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1940100650 0x7f194010af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1940101840 0x7f1940109550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1940102200 0x7f1940109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f194010a1f0 con 0x7f1940100650 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f194010a360 con 0x7f1940102200 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.800+0000 7f19452fb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f19401a1740 con 0x7f1940101840 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1940100650 0x7f194010af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1940100650 0x7f194010af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42174/0 (socket says 172.21.15.7:42174) 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193effd640 1 -- 172.21.15.7:0/2255554499 learned_addr learned my addr 172.21.15.7:0/2255554499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193f7fe640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1940102200 0x7f1940109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:39.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f1937fff640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1940101840 0x7f1940109550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193effd640 1 -- 172.21.15.7:0/2255554499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1940101840 msgr2=0x7f1940109550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193effd640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1940101840 0x7f1940109550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193effd640 1 -- 172.21.15.7:0/2255554499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1940102200 msgr2=0x7f1940109a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193effd640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1940102200 0x7f1940109a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193effd640 1 -- 172.21.15.7:0/2255554499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1928009560 con 0x7f1940100650 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193effd640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1940100650 0x7f194010af10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f193000e9f0 tx=0x7f193000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193cff9640 1 -- 172.21.15.7:0/2255554499 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930009700 con 0x7f1940100650 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f19452fb640 1 -- 172.21.15.7:0/2255554499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19401a19f0 con 0x7f1940100650 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193cff9640 1 -- 172.21.15.7:0/2255554499 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f193000cd80 con 0x7f1940100650 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f193cff9640 1 -- 172.21.15.7:0/2255554499 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930010690 con 0x7f1940100650 2024-03-23T13:19:39.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.801+0000 7f19452fb640 1 -- 172.21.15.7:0/2255554499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19401a1ed0 con 0x7f1940100650 2024-03-23T13:19:39.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.802+0000 7f19452fb640 1 -- 172.21.15.7:0/2255554499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19401078a0 con 0x7f1940100650 2024-03-23T13:19:39.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.803+0000 7f193cff9640 1 -- 172.21.15.7:0/2255554499 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f1930003710 con 0x7f1940100650 2024-03-23T13:19:39.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.804+0000 7f193cff9640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f192c0a2160 0x7f192c0a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:39.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.804+0000 7f1937fff640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f192c0a2160 0x7f192c0a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:39.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.804+0000 7f193cff9640 1 -- 172.21.15.7:0/2255554499 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1930014070 con 0x7f1940100650 2024-03-23T13:19:39.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.804+0000 7f1937fff640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f192c0a2160 0x7f192c0a4620 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f192802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:39.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:39.808+0000 7f193cff9640 1 -- 172.21.15.7:0/2255554499 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1930096f80 con 0x7f1940100650 2024-03-23T13:19:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.030+0000 7f19452fb640 1 -- 172.21.15.7:0/2255554499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f194002d230 con 0x7f1940100650 2024-03-23T13:19:40.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:39 smithi007 ceph-mon[32184]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:19:40.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:39 smithi130 ceph-mon[36225]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:19:40.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:39 smithi155 ceph-mon[36157]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-23T13:19:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.840+0000 7f193cff9640 1 -- 172.21.15.7:0/2255554499 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f193009a640 con 0x7f1940100650 2024-03-23T13:19:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 -- 172.21.15.7:0/2255554499 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f192c0a2160 msgr2=0x7f192c0a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f192c0a2160 0x7f192c0a4620 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f192802c7e0 comp rx=0 tx=0).stop 2024-03-23T13:19:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 -- 172.21.15.7:0/2255554499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1940100650 msgr2=0x7f194010af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1940100650 0x7f194010af10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f193000e9f0 tx=0x7f193000eeb0 comp rx=0 tx=0).stop 2024-03-23T13:19:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 -- 172.21.15.7:0/2255554499 shutdown_connections 2024-03-23T13:19:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f192c0a2160 0x7f192c0a4620 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1940102200 0x7f1940109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:40.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1940101840 0x7f1940109550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:40.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 --2- 172.21.15.7:0/2255554499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1940100650 0x7f194010af10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:40.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 -- 172.21.15.7:0/2255554499 >> 172.21.15.7:0/2255554499 conn(0x7f19400fbbc0 msgr2=0x7f1940104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:40.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 -- 172.21.15.7:0/2255554499 shutdown_connections 2024-03-23T13:19:40.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:19:40.843+0000 7f19452fb640 1 -- 172.21.15.7:0/2255554499 wait complete. 2024-03-23T13:19:40.858 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-23T13:19:40.869 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-23T13:19:40.945 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-23T13:19:40.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-23T13:19:40.948 INFO:teuthology.run_tasks:Running task workunit... 2024-03-23T13:19:41.067 INFO:tasks.workunit:Pulling workunits from ref d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b 2024-03-23T13:19:41.067 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-23T13:19:41.067 INFO:tasks.workunit:timeout=3h 2024-03-23T13:19:41.067 INFO:tasks.workunit:cleanup=True 2024-03-23T13:19:41.068 DEBUG:teuthology.orchestra.run.smithi007:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-23T13:19:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-23T13:19:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-23T13:19:41.084 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-23T13:19:41.084 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T13:19:41.085 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-23T13:19:41.085 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-23 13:17:49.349262686 +0000 2024-03-23T13:19:41.085 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-23 13:17:49.349262686 +0000 2024-03-23T13:19:41.085 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-23 13:19:39.346964634 +0000 2024-03-23T13:19:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-23 13:17:49.349262686 +0000 2024-03-23T13:19:41.085 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-23T13:19:41.085 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-23T13:19:41.150 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:40 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2255554499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T13:19:41.165 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b 2024-03-23T13:19:41.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:40 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2255554499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T13:19:41.222 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-23T13:19:41.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:40 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2255554499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T13:19:42.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:41 smithi130 ceph-mon[36225]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:19:42.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:41 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2255554499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-23T13:19:42.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:41 smithi130 ceph-mon[36225]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:19:42.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:41 smithi155 ceph-mon[36157]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:19:42.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:41 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2255554499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-23T13:19:42.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:41 smithi155 ceph-mon[36157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:19:42.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:41 smithi007 ceph-mon[32184]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:19:42.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:41 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2255554499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-23T13:19:42.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:41 smithi007 ceph-mon[32184]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:19:44.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:43 smithi130 ceph-mon[36225]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:19:44.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:43 smithi155 ceph-mon[36157]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:19:44.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:43 smithi007 ceph-mon[32184]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:19:46.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:45 smithi130 ceph-mon[36225]: pgmap v348: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-23T13:19:46.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:45 smithi155 ceph-mon[36157]: pgmap v348: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-23T13:19:46.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:45 smithi007 ceph-mon[32184]: pgmap v348: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-23T13:19:48.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:47 smithi007 ceph-mon[32184]: pgmap v349: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-23T13:19:48.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:47 smithi130 ceph-mon[36225]: pgmap v349: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-23T13:19:48.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:47 smithi155 ceph-mon[36157]: pgmap v349: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-23T13:19:50.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:49 smithi130 ceph-mon[36225]: pgmap v350: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-03-23T13:19:50.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:49 smithi155 ceph-mon[36157]: pgmap v350: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-03-23T13:19:50.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:49 smithi007 ceph-mon[32184]: pgmap v350: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-03-23T13:19:52.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:51 smithi130 ceph-mon[36225]: pgmap v351: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-23T13:19:52.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:51 smithi155 ceph-mon[36157]: pgmap v351: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-23T13:19:52.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:51 smithi007 ceph-mon[32184]: pgmap v351: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-23T13:19:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.086+0000 7f7ce61ac640 1 -- 172.21.15.7:0/2952670348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce0100630 msgr2=0x7f7ce0100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.086+0000 7f7ce61ac640 1 --2- 172.21.15.7:0/2952670348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce0100630 0x7f7ce0100a50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009900 tx=0x7f7cd002bc30 comp rx=0 tx=0).stop 2024-03-23T13:19:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.086+0000 7f7ce61ac640 1 -- 172.21.15.7:0/2952670348 shutdown_connections 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.086+0000 7f7ce61ac640 1 --2- 172.21.15.7:0/2952670348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ce01021e0 0x7f7ce0108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.086+0000 7f7ce61ac640 1 --2- 172.21.15.7:0/2952670348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce0101820 0x7f7ce0101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.086+0000 7f7ce61ac640 1 --2- 172.21.15.7:0/2952670348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce0100630 0x7f7ce0100a50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.086+0000 7f7ce61ac640 1 -- 172.21.15.7:0/2952670348 >> 172.21.15.7:0/2952670348 conn(0x7f7ce00fbba0 msgr2=0x7f7ce00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.087+0000 7f7ce61ac640 1 -- 172.21.15.7:0/2952670348 shutdown_connections 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.087+0000 7f7ce61ac640 1 -- 172.21.15.7:0/2952670348 wait complete. 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.087+0000 7f7ce61ac640 1 Processor -- start 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.087+0000 7f7ce61ac640 1 -- start start 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.087+0000 7f7ce61ac640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ce0100630 0x7f7ce0197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7ce61ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce0101820 0x7f7ce0197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7ce61ac640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce01021e0 0x7f7ce019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7ce61ac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce019f280 con 0x7f7ce0101820 2024-03-23T13:19:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7ce61ac640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce019f3f0 con 0x7f7ce0100630 2024-03-23T13:19:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7ce61ac640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce019f5b0 con 0x7f7ce01021e0 2024-03-23T13:19:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce0101820 0x7f7ce0197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce0101820 0x7f7ce0197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46394/0 (socket says 172.21.15.7:46394) 2024-03-23T13:19:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdeffd640 1 -- 172.21.15.7:0/220393997 learned_addr learned my addr 172.21.15.7:0/220393997 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdffff640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce01021e0 0x7f7ce019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdf7fe640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ce0100630 0x7f7ce0197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdeffd640 1 -- 172.21.15.7:0/220393997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce01021e0 msgr2=0x7f7ce019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce01021e0 0x7f7ce019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdeffd640 1 -- 172.21.15.7:0/220393997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ce0100630 msgr2=0x7f7ce0197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ce0100630 0x7f7ce0197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdeffd640 1 -- 172.21.15.7:0/220393997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd0009560 con 0x7f7ce0101820 2024-03-23T13:19:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdffff640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce01021e0 0x7f7ce019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdf7fe640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ce0100630 0x7f7ce0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce0101820 0x7f7ce0197cd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400e9f0 tx=0x7f7cd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdcff9640 1 -- 172.21.15.7:0/220393997 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4009720 con 0x7f7ce0101820 2024-03-23T13:19:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdcff9640 1 -- 172.21.15.7:0/220393997 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cd400cd80 con 0x7f7ce0101820 2024-03-23T13:19:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7ce61ac640 1 -- 172.21.15.7:0/220393997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce019f890 con 0x7f7ce0101820 2024-03-23T13:19:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7cdcff9640 1 -- 172.21.15.7:0/220393997 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4010690 con 0x7f7ce0101820 2024-03-23T13:19:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.088+0000 7f7ce61ac640 1 -- 172.21.15.7:0/220393997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce019fda0 con 0x7f7ce0101820 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7cdcff9640 1 -- 172.21.15.7:0/220393997 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f7cd40046c0 con 0x7f7ce0101820 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7ce8d89740 1 Processor -- start 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7ce8d89740 1 -- start start 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 0x555c4a077020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a07c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a07c760 0x555c4a07ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555c4a02e1b0 con 0x555c4a076c00 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555c49fe0f80 con 0x555c4a07c760 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555c49fde4b0 con 0x555c4a0775f0 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7cdcff9640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7cc80a21e0 0x7f7cc80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7cdf7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a07c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7cdf7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a07c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:41402/0 (socket says 172.21.15.7:41402) 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7cdffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 0x555c4a077020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7cdf7fe640 1 -- 172.21.15.7:0/3458649396 learned_addr learned my addr 172.21.15.7:0/3458649396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.091+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/3458649396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a07c760 0x555c4a07ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cdffff640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7cc80a21e0 0x7f7cc80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cdcff9640 1 -- 172.21.15.7:0/220393997 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7cd4014070 con 0x7f7ce0101820 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cdeffd640 1 -- 172.21.15.7:0/3458649396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 msgr2=0x555c4a07c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/3458649396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a07c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cdeffd640 1 -- 172.21.15.7:0/3458649396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 msgr2=0x555c4a077020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/3458649396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 0x555c4a077020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cdeffd640 1 -- 172.21.15.7:0/3458649396 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c49f6f810 con 0x555c4a07c760 2024-03-23T13:19:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cdf7fe640 1 --2- 172.21.15.7:0/3458649396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a07c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/3458649396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a07c760 0x555c4a07ebf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4097130 tx=0x7f7cd4005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9445875e708aeed server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cdffff640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7cc80a21e0 0x7f7cc80a46a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80099e0 tx=0x7f7cd8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cc5ffb640 1 -- 172.21.15.7:0/3458649396 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd409b3d0 con 0x555c4a07c760 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cc5ffb640 1 -- 172.21.15.7:0/3458649396 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cd409acd0 con 0x555c4a07c760 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.092+0000 7f7cc5ffb640 1 -- 172.21.15.7:0/3458649396 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd409af30 con 0x555c4a07c760 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.093+0000 7f7ce8d89740 1 -- 172.21.15.7:0/3458649396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a07c760 msgr2=0x555c4a07ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.093+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/3458649396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a07c760 0x555c4a07ebf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4097130 tx=0x7f7cd4005720 comp rx=0 tx=0).stop 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.093+0000 7f7ce8d89740 1 -- 172.21.15.7:0/3458649396 shutdown_connections 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.093+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/3458649396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a07c760 0x555c4a07ebf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.093+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/3458649396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a07c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.093+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/3458649396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 0x555c4a077020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.093+0000 7f7ce8d89740 1 -- 172.21.15.7:0/3458649396 >> 172.21.15.7:0/3458649396 conn(0x555c4a06dd60 msgr2=0x555c4a06e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.093+0000 7f7ce8d89740 1 -- 172.21.15.7:0/3458649396 shutdown_connections 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.093+0000 7f7ce8d89740 1 -- 172.21.15.7:0/3458649396 wait complete. 2024-03-23T13:19:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.093+0000 7f7ce8d89740 1 -- start start 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7ce8d89740 1 Processor -- start 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 0x555c4a0799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a079f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a07af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555c49fd9c90 con 0x555c4a07c760 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555c49f7ec50 con 0x555c4a076c00 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555c49cad250 con 0x555c4a0775f0 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7ce8d89740 10 client.0 _ll_register_callbacks cb 0x555c49fcd290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdeffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a079f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdf7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 0x555c4a0799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a07af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdeffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a079f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:41408/0 (socket says 172.21.15.7:41408) 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a07af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46406/0 (socket says 172.21.15.7:46406) 2024-03-23T13:19:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdf7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 0x555c4a0799d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:45370/0 (socket says 172.21.15.7:45370) 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdeffd640 1 -- 172.21.15.7:0/1434913879 learned_addr learned my addr 172.21.15.7:0/1434913879 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdeffd640 1 -- 172.21.15.7:0/1434913879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 msgr2=0x555c4a0799d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 0x555c4a0799d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdeffd640 1 -- 172.21.15.7:0/1434913879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 msgr2=0x555c4a07af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a07af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdeffd640 1 -- 172.21.15.7:0/1434913879 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce019fda0 con 0x555c4a0775f0 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.094+0000 7f7cdffff640 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a07af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a079f10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400e9c0 tx=0x7f7cd409b450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9072faf89a48f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7cbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7cbeffd640 1 -- 172.21.15.7:0/1434913879 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd409b130 con 0x555c4a0775f0 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7ce8d89740 1 -- 172.21.15.7:0/1434913879 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c49ffe790 con 0x555c4a0775f0 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7cbeffd640 1 -- 172.21.15.7:0/1434913879 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7cd409ee20 con 0x555c4a0775f0 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7cbeffd640 1 -- 172.21.15.7:0/1434913879 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7cd40b91d0 con 0x555c4a0775f0 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7ce8d89740 10 client.25219 fetch_fsmap learned FSMap version 27 2024-03-23T13:19:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7ce8d89740 1 -- 172.21.15.7:0/1434913879 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555c49fd8310 con 0x555c4a0775f0 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7cbeffd640 1 -- 172.21.15.7:0/1434913879 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555c49ffe790 con 0x555c4a0775f0 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7cbeffd640 1 -- 172.21.15.7:0/1434913879 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7cd40aed20 con 0x555c4a0775f0 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7ce8d89740 10 client.25219 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7ce8d89740 10 client.25219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7ce8d89740 20 client.25219 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x555c4a0884d0 0x555c4a08a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7ce8d89740 4 client.25219 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.095+0000 7f7ce8d89740 1 -- 172.21.15.7:0/1434913879 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c49cd9600 con 0x555c4a0884d0 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.096+0000 7f7cdf7fe640 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x555c4a0884d0 0x555c4a08a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.096+0000 7f7cbeffd640 10 client.25219 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.096+0000 7f7cdf7fe640 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x555c4a0884d0 0x555c4a08a8c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e391311c141d41a8 server_cookie=cd49a9c2892cfcc9 in_seq=0 out_seq=0 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7cbeffd640 1 -- 172.21.15.7:0/1434913879 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555c49cd9600 con 0x555c4a0884d0 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7cbeffd640 10 client.25219 handle_command_reply: tid=0 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 client.25219 shutdown 2024-03-23T13:19:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 -- 172.21.15.7:0/1434913879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 msgr2=0x555c4a079f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a079f10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400e9c0 tx=0x7f7cd409b450 comp rx=0 tx=0).stop 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 -- 172.21.15.7:0/1434913879 shutdown_connections 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x555c4a0884d0 0x555c4a08a8c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a07af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7cbeffd640 0 client.25219 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a079f10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/1434913879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 0x555c4a0799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 -- 172.21.15.7:0/1434913879 >> 172.21.15.7:0/1434913879 conn(0x555c4a06dd60 msgr2=0x555c4a06cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 -- 172.21.15.7:0/1434913879 shutdown_connections 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 1 -- 172.21.15.7:0/1434913879 wait complete. 2024-03-23T13:19:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.097+0000 7f7ce8d89740 20 client.25219 trim_cache size 0 max 16384 2024-03-23T13:19:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.112+0000 7f7ce8d89740 1 Processor -- start 2024-03-23T13:19:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.112+0000 7f7ce8d89740 1 -- start start 2024-03-23T13:19:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.112+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 0x555c4a0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.112+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a073810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a078090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555c49f5f660 con 0x555c4a07c760 2024-03-23T13:19:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555c49cd9600 con 0x555c4a076c00 2024-03-23T13:19:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555c4a077ce0 con 0x555c4a0775f0 2024-03-23T13:19:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a078090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a078090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46414/0 (socket says 172.21.15.7:46414) 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdf7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 0x555c4a0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdffff640 1 -- 172.21.15.7:0/757437409 learned_addr learned my addr 172.21.15.7:0/757437409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/757437409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a073810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdffff640 1 -- 172.21.15.7:0/757437409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 msgr2=0x555c4a073810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdffff640 1 --2- 172.21.15.7:0/757437409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a073810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdffff640 1 -- 172.21.15.7:0/757437409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 msgr2=0x555c4a0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdffff640 1 --2- 172.21.15.7:0/757437409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 0x555c4a0732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdffff640 1 -- 172.21.15.7:0/757437409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c49f74810 con 0x555c4a07c760 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdf7fe640 1 --2- 172.21.15.7:0/757437409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 0x555c4a0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:19:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cdffff640 1 --2- 172.21.15.7:0/757437409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a078090 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800fe70 tx=0x7f7cd800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68d83baaf538d80 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cc57fa640 1 -- 172.21.15.7:0/757437409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8026040 con 0x555c4a07c760 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cc57fa640 1 -- 172.21.15.7:0/757437409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cd8011020 con 0x555c4a07c760 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.113+0000 7f7cc57fa640 1 -- 172.21.15.7:0/757437409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd801f6c0 con 0x555c4a07c760 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.114+0000 7f7ce8d89740 1 -- 172.21.15.7:0/757437409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 msgr2=0x555c4a078090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.114+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/757437409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a078090 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800fe70 tx=0x7f7cd800e000 comp rx=0 tx=0).stop 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.114+0000 7f7ce8d89740 1 -- 172.21.15.7:0/757437409 shutdown_connections 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.114+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/757437409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a07c760 0x555c4a078090 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.114+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/757437409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a0775f0 0x555c4a073810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.114+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/757437409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a076c00 0x555c4a0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.114+0000 7f7ce8d89740 1 -- 172.21.15.7:0/757437409 >> 172.21.15.7:0/757437409 conn(0x555c4a06dd60 msgr2=0x555c4a06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.114+0000 7f7ce8d89740 1 -- 172.21.15.7:0/757437409 shutdown_connections 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.114+0000 7f7ce8d89740 1 -- 172.21.15.7:0/757437409 wait complete. 2024-03-23T13:19:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7ce8d89740 1 -- start start 2024-03-23T13:19:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7ce8d89740 1 Processor -- start 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 0x555c4a082210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a0775f0 0x555c4a082750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7ce8d89740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a07c760 0x555c4a083760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555c4a083ca0 con 0x555c4a076c00 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555c4a083e10 con 0x555c4a0775f0 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7ce8d89740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555c4a083fd0 con 0x555c4a07c760 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7ce8d89740 10 client.0 _ll_register_callbacks cb 0x7f7cd8011760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7cdf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 0x555c4a082210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7cdf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 0x555c4a082210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46416/0 (socket says 172.21.15.7:46416) 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7cdf7fe640 1 -- 172.21.15.7:0/4206502825 learned_addr learned my addr 172.21.15.7:0/4206502825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:19:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a0775f0 0x555c4a082750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.115+0000 7f7cdffff640 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a07c760 0x555c4a083760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cdf7fe640 1 -- 172.21.15.7:0/4206502825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a07c760 msgr2=0x555c4a083760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cdf7fe640 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a07c760 0x555c4a083760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cdf7fe640 1 -- 172.21.15.7:0/4206502825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a0775f0 msgr2=0x555c4a082750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cdf7fe640 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a0775f0 0x555c4a082750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cdf7fe640 1 -- 172.21.15.7:0/4206502825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd0030070 con 0x555c4a076c00 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cdf7fe640 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 0x555c4a082210 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0036040 tx=0x7f7cd0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d07d0f0442e8afc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cbeffd640 1 -- 172.21.15.7:0/4206502825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd004a3f0 con 0x555c4a076c00 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cbeffd640 1 -- 172.21.15.7:0/4206502825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7cd0046040 con 0x555c4a076c00 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7ce8d89740 1 -- 172.21.15.7:0/4206502825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cd409acd0 con 0x555c4a076c00 2024-03-23T13:19:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cbeffd640 1 -- 172.21.15.7:0/4206502825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7cd004a550 con 0x555c4a076c00 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7ce8d89740 10 client.15375 fetch_fsmap learned FSMap version 27 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7ce8d89740 1 -- 172.21.15.7:0/4206502825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x7f7cd8019040 con 0x555c4a076c00 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.116+0000 7f7cbeffd640 1 -- 172.21.15.7:0/4206502825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cd409acd0 con 0x555c4a076c00 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.117+0000 7f7cbeffd640 1 -- 172.21.15.7:0/4206502825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7cd0053020 con 0x555c4a076c00 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.117+0000 7f7ce8d89740 10 client.15375 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.117+0000 7f7ce8d89740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.117+0000 7f7ce8d89740 20 client.15375 populate_metadata read hostname 'smithi007' 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.117+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x555c4a0eed80 0x555c4a0f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.117+0000 7f7ce8d89740 4 client.15375 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.117+0000 7f7ce8d89740 1 -- 172.21.15.7:0/4206502825 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c49d2a1b0 con 0x555c4a0eed80 2024-03-23T13:19:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.117+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x555c4a0eed80 0x555c4a0f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:19:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.117+0000 7f7cdeffd640 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x555c4a0eed80 0x555c4a0f11a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e791837ee65dbf61 server_cookie=ca3d47f3d18bc6d2 in_seq=0 out_seq=0 2024-03-23T13:19:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.117+0000 7f7cbeffd640 10 client.15375 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7cbeffd640 1 -- 172.21.15.7:0/4206502825 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c49d2a1b0 con 0x555c4a0eed80 2024-03-23T13:19:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7cbeffd640 10 client.15375 handle_command_reply: tid=0 2024-03-23T13:19:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7ce8d89740 1 client.15375 shutdown 2024-03-23T13:19:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7ce8d89740 1 -- 172.21.15.7:0/4206502825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 msgr2=0x555c4a082210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 0x555c4a082210 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0036040 tx=0x7f7cd0004ad0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7ce8d89740 1 -- 172.21.15.7:0/4206502825 shutdown_connections 2024-03-23T13:19:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x555c4a0eed80 0x555c4a0f11a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4a07c760 0x555c4a083760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7cbeffd640 0 client.15375 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:19:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555c4a0775f0 0x555c4a082750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7ce8d89740 1 --2- 172.21.15.7:0/4206502825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c4a076c00 0x555c4a082210 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7ce8d89740 1 -- 172.21.15.7:0/4206502825 >> 172.21.15.7:0/4206502825 conn(0x555c4a06dd60 msgr2=0x555c4a078090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.127+0000 7f7ce8d89740 1 -- 172.21.15.7:0/4206502825 shutdown_connections 2024-03-23T13:19:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce8d89740 1 -- 172.21.15.7:0/4206502825 wait complete. 2024-03-23T13:19:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce8d89740 20 client.15375 trim_cache size 0 max 16384 2024-03-23T13:19:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-23T13:19:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-23T13:19:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-23T13:19:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-23T13:19:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 -- 172.21.15.7:0/220393997 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7cc80a21e0 msgr2=0x7f7cc80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7cc80a21e0 0x7f7cc80a46a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80099e0 tx=0x7f7cd8009500 comp rx=0 tx=0).stop 2024-03-23T13:19:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 -- 172.21.15.7:0/220393997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce0101820 msgr2=0x7f7ce0197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:19:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce0101820 0x7f7ce0197cd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400e9f0 tx=0x7f7cd400edf0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 -- 172.21.15.7:0/220393997 shutdown_connections 2024-03-23T13:19:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7cc80a21e0 0x7f7cc80a46a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce01021e0 0x7f7ce019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce0101820 0x7f7ce0197cd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 --2- 172.21.15.7:0/220393997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ce0100630 0x7f7ce0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:19:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 -- 172.21.15.7:0/220393997 >> 172.21.15.7:0/220393997 conn(0x7f7ce00fbba0 msgr2=0x7f7ce00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:19:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 -- 172.21.15.7:0/220393997 shutdown_connections 2024-03-23T13:19:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:19:54.128+0000 7f7ce61ac640 1 -- 172.21.15.7:0/220393997 wait complete. 2024-03-23T13:19:54.198 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:53 smithi130 ceph-mon[36225]: pgmap v352: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-23T13:19:54.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:53 smithi155 ceph-mon[36157]: pgmap v352: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-23T13:19:54.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:53 smithi007 ceph-mon[32184]: pgmap v352: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-23T13:19:56.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:55 smithi130 ceph-mon[36225]: pgmap v353: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T13:19:56.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:55 smithi155 ceph-mon[36157]: pgmap v353: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T13:19:56.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:55 smithi007 ceph-mon[32184]: pgmap v353: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T13:19:58.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:57 smithi007 ceph-mon[32184]: pgmap v354: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:19:58.195 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:57 smithi130 ceph-mon[36225]: pgmap v354: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:19:58.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:57 smithi155 ceph-mon[36157]: pgmap v354: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T13:20:00.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:19:59 smithi130 ceph-mon[36225]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-23T13:20:00.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:19:59 smithi155 ceph-mon[36157]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-23T13:20:00.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:19:59 smithi007 ceph-mon[32184]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-23T13:20:01.031 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:00 smithi130 ceph-mon[36225]: overall HEALTH_OK 2024-03-23T13:20:01.031 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:00 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:20:01.205 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:00 smithi155 ceph-mon[36157]: overall HEALTH_OK 2024-03-23T13:20:01.205 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:00 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:20:01.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:00 smithi007 ceph-mon[32184]: overall HEALTH_OK 2024-03-23T13:20:01.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:00 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:20:02.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:01 smithi130 ceph-mon[36225]: pgmap v356: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-23T13:20:02.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:01 smithi155 ceph-mon[36157]: pgmap v356: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-23T13:20:02.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:01 smithi007 ceph-mon[32184]: pgmap v356: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-23T13:20:03.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:02 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:02 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:02 smithi130 ceph-mon[36225]: pgmap v357: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-03-23T13:20:03.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:02 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:02 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.197 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:02 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.197 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:02 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:02 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:02 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:02 smithi155 ceph-mon[36157]: pgmap v357: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-03-23T13:20:03.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:02 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:02 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:02 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:02 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:02 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:02 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:02 smithi007 ceph-mon[32184]: pgmap v357: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-03-23T13:20:03.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:02 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:02 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:02 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:03.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:02 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:04.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:20:04.196 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:20:04.197 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:03 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:04.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:20:04.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:20:04.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:03 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:04.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:20:04.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:20:04.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:03 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:20:05.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:04 smithi155 ceph-mon[36157]: pgmap v358: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-03-23T13:20:05.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:04 smithi007 ceph-mon[32184]: pgmap v358: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-03-23T13:20:05.417 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:04 smithi130 ceph-mon[36225]: pgmap v358: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-03-23T13:20:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.256+0000 7f7f6129d640 1 -- 172.21.15.7:0/2118464308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5c1021b0 msgr2=0x7f7f5c1045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.256+0000 7f7f6129d640 1 --2- 172.21.15.7:0/2118464308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5c1021b0 0x7f7f5c1045a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009900 tx=0x7f7f4c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:20:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.256+0000 7f7f6129d640 1 -- 172.21.15.7:0/2118464308 shutdown_connections 2024-03-23T13:20:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.256+0000 7f7f6129d640 1 --2- 172.21.15.7:0/2118464308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c107410 0x7f7f5c109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.256+0000 7f7f6129d640 1 --2- 172.21.15.7:0/2118464308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c104ae0 0x7f7f5c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.256+0000 7f7f6129d640 1 --2- 172.21.15.7:0/2118464308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5c1021b0 0x7f7f5c1045a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.256+0000 7f7f6129d640 1 -- 172.21.15.7:0/2118464308 >> 172.21.15.7:0/2118464308 conn(0x7f7f5c0fbba0 msgr2=0x7f7f5c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.257+0000 7f7f6129d640 1 -- 172.21.15.7:0/2118464308 shutdown_connections 2024-03-23T13:20:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.257+0000 7f7f6129d640 1 -- 172.21.15.7:0/2118464308 wait complete. 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.257+0000 7f7f6129d640 1 Processor -- start 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.257+0000 7f7f6129d640 1 -- start start 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.257+0000 7f7f6129d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c1021b0 0x7f7f5c101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.257+0000 7f7f6129d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c104ae0 0x7f7f5c100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.257+0000 7f7f6129d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5c107410 0x7f7f5c1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f6129d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c100e00 con 0x7f7f5c107410 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f6129d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c100f70 con 0x7f7f5c104ae0 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f6129d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1a17a0 con 0x7f7f5c1021b0 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5a7fc640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c104ae0 0x7f7f5c100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c1021b0 0x7f7f5c101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5a7fc640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c104ae0 0x7f7f5c100160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:44288/0 (socket says 172.21.15.7:44288) 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5a7fc640 1 -- 172.21.15.7:0/3684611326 learned_addr learned my addr 172.21.15.7:0/3684611326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c1021b0 0x7f7f5c101b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:58386/0 (socket says 172.21.15.7:58386) 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5b7fe640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5c107410 0x7f7f5c1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5affd640 1 -- 172.21.15.7:0/3684611326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c104ae0 msgr2=0x7f7f5c100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5affd640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c104ae0 0x7f7f5c100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5affd640 1 -- 172.21.15.7:0/3684611326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5c107410 msgr2=0x7f7f5c1006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5affd640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5c107410 0x7f7f5c1006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5affd640 1 -- 172.21.15.7:0/3684611326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c009560 con 0x7f7f5c1021b0 2024-03-23T13:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f5affd640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c1021b0 0x7f7f5c101b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000c00 tx=0x7f7f4c005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.258+0000 7f7f43fff640 1 -- 172.21.15.7:0/3684611326 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0043d0 con 0x7f7f5c1021b0 2024-03-23T13:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.259+0000 7f7f6129d640 1 -- 172.21.15.7:0/3684611326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5c1a19f0 con 0x7f7f5c1021b0 2024-03-23T13:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.259+0000 7f7f43fff640 1 -- 172.21.15.7:0/3684611326 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c0052d0 con 0x7f7f5c1021b0 2024-03-23T13:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.259+0000 7f7f43fff640 1 -- 172.21.15.7:0/3684611326 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c003d40 con 0x7f7f5c1021b0 2024-03-23T13:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.259+0000 7f7f6129d640 1 -- 172.21.15.7:0/3684611326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5c1a1ed0 con 0x7f7f5c1021b0 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.261+0000 7f7f43fff640 1 -- 172.21.15.7:0/3684611326 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f7f4c005440 con 0x7f7f5c1021b0 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.261+0000 7f7f43fff640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7f440a2160 0x7f7f440a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f43fff640 1 -- 172.21.15.7:0/3684611326 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7f4c0efd30 con 0x7f7f5c1021b0 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7f440a2160 0x7f7f440a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f63e82740 1 Processor -- start 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f63e82740 1 -- start start 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c2490c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24911e0 0x5638c2495ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 0x5638c24987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f63e82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5638c244c1b0 con 0x5638c24907f0 2024-03-23T13:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f63e82740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5638c23fef80 con 0x5638c24911e0 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f63e82740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638c23fc4b0 con 0x5638c2496350 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c2490c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c2490c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49768/0 (socket says 172.21.15.7:49768) 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f5affd640 1 -- 172.21.15.7:0/1939775286 learned_addr learned my addr 172.21.15.7:0/1939775286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f5b7fe640 1 --2- 172.21.15.7:0/1939775286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 0x5638c24987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.262+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7f440a2160 0x7f7f440a4620 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c101890 tx=0x7f7f50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.263+0000 7f7f5affd640 1 -- 172.21.15.7:0/1939775286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 msgr2=0x5638c24987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.263+0000 7f7f5affd640 1 --2- 172.21.15.7:0/1939775286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 0x5638c24987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.263+0000 7f7f5affd640 1 -- 172.21.15.7:0/1939775286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24911e0 msgr2=0x5638c2495ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.263+0000 7f7f5affd640 1 --2- 172.21.15.7:0/1939775286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24911e0 0x5638c2495ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.263+0000 7f7f5affd640 1 -- 172.21.15.7:0/1939775286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638c238d810 con 0x5638c24907f0 2024-03-23T13:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.263+0000 7f7f5affd640 1 --2- 172.21.15.7:0/1939775286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c2490c10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0bc770 tx=0x7f7f4c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a914cd5685195921 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.263+0000 7f7f417fa640 1 -- 172.21.15.7:0/1939775286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0bca70 con 0x5638c24907f0 2024-03-23T13:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.263+0000 7f7f417fa640 1 -- 172.21.15.7:0/1939775286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c0c0bc0 con 0x5638c24907f0 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.263+0000 7f7f417fa640 1 -- 172.21.15.7:0/1939775286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0bae70 con 0x5638c24907f0 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 -- 172.21.15.7:0/1939775286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 msgr2=0x5638c2490c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 --2- 172.21.15.7:0/1939775286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c2490c10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0bc770 tx=0x7f7f4c005fd0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 -- 172.21.15.7:0/1939775286 shutdown_connections 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 --2- 172.21.15.7:0/1939775286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 0x5638c24987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 --2- 172.21.15.7:0/1939775286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24911e0 0x5638c2495ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 --2- 172.21.15.7:0/1939775286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c2490c10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 -- 172.21.15.7:0/1939775286 >> 172.21.15.7:0/1939775286 conn(0x5638c248bd60 msgr2=0x5638c248c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 -- 172.21.15.7:0/1939775286 shutdown_connections 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 -- 172.21.15.7:0/1939775286 wait complete. 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 -- start start 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.264+0000 7f7f63e82740 1 Processor -- start 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c249a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 0x5638c249aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c2496350 0x5638c24a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f63e82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5638c23f7c90 con 0x5638c24907f0 2024-03-23T13:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c249a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f63e82740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5638c239cc50 con 0x5638c2496350 2024-03-23T13:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f63e82740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638c20cb250 con 0x5638c24911e0 2024-03-23T13:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c249a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49776/0 (socket says 172.21.15.7:49776) 2024-03-23T13:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5affd640 1 -- 172.21.15.7:0/3119825778 learned_addr learned my addr 172.21.15.7:0/3119825778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 0x5638c249aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f63e82740 10 client.0 _ll_register_callbacks cb 0x5638c23eb290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5b7fe640 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c2496350 0x5638c24a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5affd640 1 -- 172.21.15.7:0/3119825778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 msgr2=0x5638c249aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5affd640 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 0x5638c249aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5affd640 1 -- 172.21.15.7:0/3119825778 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c2496350 msgr2=0x5638c24a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5affd640 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c2496350 0x5638c24a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5affd640 1 -- 172.21.15.7:0/3119825778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5c1a1ed0 con 0x5638c24907f0 2024-03-23T13:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5b7fe640 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c2496350 0x5638c24a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f5affd640 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c249a560 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0b9440 tx=0x7f7f4c0baed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea441a92114c26a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3119825778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0bca70 con 0x5638c24907f0 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f63e82740 1 -- 172.21.15.7:0/3119825778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638c241c790 con 0x5638c24907f0 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.265+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3119825778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7f4c0c4d80 con 0x5638c24907f0 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3119825778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7f4c0c4f40 con 0x5638c24907f0 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f63e82740 10 client.15387 fetch_fsmap learned FSMap version 27 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f63e82740 1 -- 172.21.15.7:0/3119825778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638c23f6310 con 0x5638c24907f0 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3119825778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c09f080 con 0x5638c24907f0 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3119825778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7f4c03c070 con 0x5638c24907f0 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f63e82740 10 client.15387 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f63e82740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f63e82740 20 client.15387 populate_metadata read hostname 'smithi007' 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5638c24ae7a0 0x5638c24b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f63e82740 4 client.15387 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f63e82740 1 -- 172.21.15.7:0/3119825778 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638c20f7600 con 0x5638c24ae7a0 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.266+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5638c24ae7a0 0x5638c24b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.267+0000 7f7f3eff5640 10 client.15387 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.267+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5638c24ae7a0 0x5638c24b0b90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a7ba3e30c9c3e16 server_cookie=55c0a2535cc10532 in_seq=0 out_seq=0 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.267+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3119825778 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5638c20f7600 con 0x5638c24ae7a0 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.267+0000 7f7f3eff5640 10 client.15387 handle_command_reply: tid=0 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.267+0000 7f7f63e82740 1 client.15387 shutdown 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 1 -- 172.21.15.7:0/3119825778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 msgr2=0x5638c249a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c249a560 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0b9440 tx=0x7f7f4c0baed0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 1 -- 172.21.15.7:0/3119825778 shutdown_connections 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5638c24ae7a0 0x5638c24b0b90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c2496350 0x5638c24a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 0x5638c249aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f3eff5640 0 client.15387 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3119825778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c249a560 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 1 -- 172.21.15.7:0/3119825778 >> 172.21.15.7:0/3119825778 conn(0x5638c248bd60 msgr2=0x5638c248aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 1 -- 172.21.15.7:0/3119825778 shutdown_connections 2024-03-23T13:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 1 -- 172.21.15.7:0/3119825778 wait complete. 2024-03-23T13:20:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.268+0000 7f7f63e82740 20 client.15387 trim_cache size 0 max 16384 2024-03-23T13:20:07.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:07 smithi155 ceph-mon[36157]: pgmap v359: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-23T13:20:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f63e82740 1 Processor -- start 2024-03-23T13:20:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f63e82740 1 -- start start 2024-03-23T13:20:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c24b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24911e0 0x5638c24a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 0x5638c248efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f63e82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5638c237d660 con 0x5638c24907f0 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f63e82740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5638c2490480 con 0x5638c24911e0 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f63e82740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638c24876c0 con 0x5638c2496350 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c24b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f5a7fc640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24911e0 0x5638c24a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c24b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49784/0 (socket says 172.21.15.7:49784) 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f5affd640 1 -- 172.21.15.7:0/1812434056 learned_addr learned my addr 172.21.15.7:0/1812434056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.297+0000 7f7f5b7fe640 1 --2- 172.21.15.7:0/1812434056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 0x5638c248efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f5a7fc640 1 -- 172.21.15.7:0/1812434056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 msgr2=0x5638c248efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/1812434056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 0x5638c248efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f5a7fc640 1 -- 172.21.15.7:0/1812434056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 msgr2=0x5638c24b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/1812434056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c24b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f5a7fc640 1 -- 172.21.15.7:0/1812434056 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638c2392810 con 0x5638c24911e0 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f5affd640 1 --2- 172.21.15.7:0/1812434056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c24b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f5b7fe640 1 --2- 172.21.15.7:0/1812434056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 0x5638c248efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:20:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/1812434056 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24911e0 0x5638c24a1a70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x5638c24b13f0 tx=0x7f7f50014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db39f282061c2e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f40ff9640 1 -- 172.21.15.7:0/1812434056 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50024070 con 0x5638c24911e0 2024-03-23T13:20:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f40ff9640 1 -- 172.21.15.7:0/1812434056 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f50025040 con 0x5638c24911e0 2024-03-23T13:20:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.298+0000 7f7f40ff9640 1 -- 172.21.15.7:0/1812434056 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5001e650 con 0x5638c24911e0 2024-03-23T13:20:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 -- 172.21.15.7:0/1812434056 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24911e0 msgr2=0x5638c24a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 --2- 172.21.15.7:0/1812434056 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24911e0 0x5638c24a1a70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x5638c24b13f0 tx=0x7f7f50014000 comp rx=0 tx=0).stop 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 -- 172.21.15.7:0/1812434056 shutdown_connections 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 --2- 172.21.15.7:0/1812434056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c2496350 0x5638c248efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 --2- 172.21.15.7:0/1812434056 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24911e0 0x5638c24a1a70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 --2- 172.21.15.7:0/1812434056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c24907f0 0x5638c24b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 -- 172.21.15.7:0/1812434056 >> 172.21.15.7:0/1812434056 conn(0x5638c248bd60 msgr2=0x5638c248c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 -- 172.21.15.7:0/1812434056 shutdown_connections 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 -- 172.21.15.7:0/1812434056 wait complete. 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 -- start start 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 Processor -- start 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24907f0 0x5638c249f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 0x5638c249f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.299+0000 7f7f63e82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c2496350 0x5638c24a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f63e82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5638c24a0b20 con 0x5638c2496350 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f63e82740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5638c24a0c90 con 0x5638c24907f0 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f63e82740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638c24a0e50 con 0x5638c24911e0 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f63e82740 10 client.0 _ll_register_callbacks cb 0x7f7f54009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24907f0 0x5638c249f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c2496350 0x5638c24a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24907f0 0x5638c249f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:44324/0 (socket says 172.21.15.7:44324) 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5affd640 1 -- 172.21.15.7:0/3332666220 learned_addr learned my addr 172.21.15.7:0/3332666220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 0x5638c249f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5affd640 1 -- 172.21.15.7:0/3332666220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 msgr2=0x5638c249f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5affd640 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 0x5638c249f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5affd640 1 -- 172.21.15.7:0/3332666220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c2496350 msgr2=0x5638c24a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5affd640 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c2496350 0x5638c24a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5affd640 1 -- 172.21.15.7:0/3332666220 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638c23f6310 con 0x5638c24907f0 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 0x5638c249f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5b7fe640 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c2496350 0x5638c24a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f5affd640 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24907f0 0x5638c249f090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0b9440 tx=0x7f7f4c0b8da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a962b1b2778fc61 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3332666220 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c003ea0 con 0x5638c24907f0 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3332666220 <== mon.2 v2:172.21.15.130:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7f4c0bca70 con 0x5638c24907f0 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.300+0000 7f7f63e82740 1 -- 172.21.15.7:0/3332666220 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4c0c0bc0 con 0x5638c24907f0 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3332666220 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7f4c0bad70 con 0x5638c24907f0 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f63e82740 10 client.25211 fetch_fsmap learned FSMap version 27 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f63e82740 1 -- 172.21.15.7:0/3332666220 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x7f7f50017070 con 0x5638c24907f0 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3332666220 <== mon.2 v2:172.21.15.130:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c0c0bc0 con 0x5638c24907f0 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3332666220 <== mon.2 v2:172.21.15.130:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7f4c03c070 con 0x5638c24907f0 2024-03-23T13:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f63e82740 10 client.25211 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T13:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f63e82740 10 client.25211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f63e82740 20 client.25211 populate_metadata read hostname 'smithi007' 2024-03-23T13:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5638c2514fb0 0x5638c25173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f63e82740 4 client.25211 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T13:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f63e82740 1 -- 172.21.15.7:0/3332666220 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638c21481b0 con 0x5638c2514fb0 2024-03-23T13:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5638c2514fb0 0x5638c25173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f5a7fc640 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5638c2514fb0 0x5638c25173d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec8b532fabdca8f server_cookie=b5eab2d1af5bee7f in_seq=0 out_seq=0 2024-03-23T13:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.301+0000 7f7f3eff5640 10 client.25211 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.306+0000 7f7f3eff5640 1 -- 172.21.15.7:0/3332666220 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638c21481b0 con 0x5638c2514fb0 2024-03-23T13:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.306+0000 7f7f3eff5640 10 client.25211 handle_command_reply: tid=0 2024-03-23T13:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.306+0000 7f7f63e82740 1 client.25211 shutdown 2024-03-23T13:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 1 -- 172.21.15.7:0/3332666220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24907f0 msgr2=0x5638c249f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24907f0 0x5638c249f090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0b9440 tx=0x7f7f4c0b8da0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 1 -- 172.21.15.7:0/3332666220 shutdown_connections 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5638c2514fb0 0x5638c25173d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f3eff5640 0 client.25211 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638c2496350 0x5638c24a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c24911e0 0x5638c249f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 1 --2- 172.21.15.7:0/3332666220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5638c24907f0 0x5638c249f090 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 1 -- 172.21.15.7:0/3332666220 >> 172.21.15.7:0/3332666220 conn(0x5638c248bd60 msgr2=0x5638c248efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 1 -- 172.21.15.7:0/3332666220 shutdown_connections 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 1 -- 172.21.15.7:0/3332666220 wait complete. 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f63e82740 20 client.25211 trim_cache size 0 max 16384 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-23T13:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-23T13:20:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-23T13:20:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f6129d640 1 -- 172.21.15.7:0/3684611326 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7f440a2160 msgr2=0x7f7f440a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.307+0000 7f7f6129d640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7f440a2160 0x7f7f440a4620 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c101890 tx=0x7f7f50005f90 comp rx=0 tx=0).stop 2024-03-23T13:20:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.308+0000 7f7f6129d640 1 -- 172.21.15.7:0/3684611326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c1021b0 msgr2=0x7f7f5c101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.308+0000 7f7f6129d640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c1021b0 0x7f7f5c101b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000c00 tx=0x7f7f4c005e50 comp rx=0 tx=0).stop 2024-03-23T13:20:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.308+0000 7f7f6129d640 1 -- 172.21.15.7:0/3684611326 shutdown_connections 2024-03-23T13:20:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.308+0000 7f7f6129d640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f7f440a2160 0x7f7f440a4620 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.308+0000 7f7f6129d640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5c107410 0x7f7f5c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.308+0000 7f7f6129d640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c104ae0 0x7f7f5c100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.308+0000 7f7f6129d640 1 --2- 172.21.15.7:0/3684611326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c1021b0 0x7f7f5c101b20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.308+0000 7f7f6129d640 1 -- 172.21.15.7:0/3684611326 >> 172.21.15.7:0/3684611326 conn(0x7f7f5c0fbba0 msgr2=0x7f7f5c102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.308+0000 7f7f6129d640 1 -- 172.21.15.7:0/3684611326 shutdown_connections 2024-03-23T13:20:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:07.308+0000 7f7f6129d640 1 -- 172.21.15.7:0/3684611326 wait complete. 2024-03-23T13:20:07.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:07 smithi007 ceph-mon[32184]: pgmap v359: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-23T13:20:07.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:07 smithi130 ceph-mon[36225]: pgmap v359: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-23T13:20:09.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:09 smithi155 ceph-mon[36157]: pgmap v360: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-03-23T13:20:09.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:09 smithi007 ceph-mon[32184]: pgmap v360: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-03-23T13:20:09.445 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:09 smithi130 ceph-mon[36225]: pgmap v360: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-03-23T13:20:11.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:11 smithi155 ceph-mon[36157]: pgmap v361: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-23T13:20:11.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:11 smithi007 ceph-mon[32184]: pgmap v361: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-23T13:20:11.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:11 smithi130 ceph-mon[36225]: pgmap v361: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-23T13:20:13.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:13 smithi155 ceph-mon[36157]: pgmap v362: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T13:20:13.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:13 smithi007 ceph-mon[32184]: pgmap v362: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T13:20:13.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:13 smithi130 ceph-mon[36225]: pgmap v362: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T13:20:15.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:15 smithi155 ceph-mon[36157]: pgmap v363: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T13:20:15.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:15 smithi007 ceph-mon[32184]: pgmap v363: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T13:20:15.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:15 smithi130 ceph-mon[36225]: pgmap v363: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T13:20:17.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:17 smithi155 ceph-mon[36157]: pgmap v364: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-23T13:20:17.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:17 smithi007 ceph-mon[32184]: pgmap v364: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-23T13:20:17.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:17 smithi130 ceph-mon[36225]: pgmap v364: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-23T13:20:19.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:19 smithi155 ceph-mon[36157]: pgmap v365: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-23T13:20:19.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:19 smithi007 ceph-mon[32184]: pgmap v365: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-23T13:20:19.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:19 smithi130 ceph-mon[36225]: pgmap v365: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-23T13:20:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.442+0000 7fa128d81640 1 -- 172.21.15.7:0/425759815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa124109820 msgr2=0x7fa12410bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.442+0000 7fa128d81640 1 --2- 172.21.15.7:0/425759815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa124109820 0x7fa12410bc10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009900 tx=0x7fa11c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:20:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.442+0000 7fa128d81640 1 -- 172.21.15.7:0/425759815 shutdown_connections 2024-03-23T13:20:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.442+0000 7fa128d81640 1 --2- 172.21.15.7:0/425759815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12410ead0 0x7fa124110f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.442+0000 7fa128d81640 1 --2- 172.21.15.7:0/425759815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa12410c150 0x7fa12410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.442+0000 7fa128d81640 1 --2- 172.21.15.7:0/425759815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa124109820 0x7fa12410bc10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.442+0000 7fa128d81640 1 -- 172.21.15.7:0/425759815 >> 172.21.15.7:0/425759815 conn(0x7fa12406e750 msgr2=0x7fa124070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.442+0000 7fa128d81640 1 -- 172.21.15.7:0/425759815 shutdown_connections 2024-03-23T13:20:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.442+0000 7fa128d81640 1 -- 172.21.15.7:0/425759815 wait complete. 2024-03-23T13:20:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.443+0000 7fa128d81640 1 Processor -- start 2024-03-23T13:20:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.443+0000 7fa128d81640 1 -- start start 2024-03-23T13:20:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.443+0000 7fa128d81640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124109820 0x7fa1241a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.443+0000 7fa128d81640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12410c150 0x7fa1241a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.443+0000 7fa128d81640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12410ead0 0x7fa1241a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.443+0000 7fa128d81640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241a9c10 con 0x7fa124109820 2024-03-23T13:20:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.443+0000 7fa128d81640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241a9d80 con 0x7fa12410c150 2024-03-23T13:20:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.443+0000 7fa128d81640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241a9f40 con 0x7fa12410ead0 2024-03-23T13:20:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa12259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124109820 0x7fa1241a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa121d9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12410c150 0x7fa1241a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa12259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124109820 0x7fa1241a2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36972/0 (socket says 172.21.15.7:36972) 2024-03-23T13:20:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa12259c640 1 -- 172.21.15.7:0/2937550523 learned_addr learned my addr 172.21.15.7:0/2937550523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa122d9d640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12410ead0 0x7fa1241a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa12259c640 1 -- 172.21.15.7:0/2937550523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12410ead0 msgr2=0x7fa1241a96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa12259c640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12410ead0 0x7fa1241a96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa12259c640 1 -- 172.21.15.7:0/2937550523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12410c150 msgr2=0x7fa1241a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa12259c640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12410c150 0x7fa1241a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa12259c640 1 -- 172.21.15.7:0/2937550523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa118009840 con 0x7fa124109820 2024-03-23T13:20:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa121d9b640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12410c150 0x7fa1241a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa122d9d640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12410ead0 0x7fa1241a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa12259c640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124109820 0x7fa1241a2120 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006010 tx=0x7fa11c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa1137fe640 1 -- 172.21.15.7:0/2937550523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0043d0 con 0x7fa124109820 2024-03-23T13:20:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.444+0000 7fa128d81640 1 -- 172.21.15.7:0/2937550523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11c009560 con 0x7fa124109820 2024-03-23T13:20:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.445+0000 7fa128d81640 1 -- 172.21.15.7:0/2937550523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12419dd00 con 0x7fa124109820 2024-03-23T13:20:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.445+0000 7fa1137fe640 1 -- 172.21.15.7:0/2937550523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11c0052d0 con 0x7fa124109820 2024-03-23T13:20:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.445+0000 7fa1137fe640 1 -- 172.21.15.7:0/2937550523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c003c50 con 0x7fa124109820 2024-03-23T13:20:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.448+0000 7fa12b5fd740 1 Processor -- start 2024-03-23T13:20:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.448+0000 7fa1137fe640 1 -- 172.21.15.7:0/2937550523 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fa11c005440 con 0x7fa124109820 2024-03-23T13:20:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.448+0000 7fa12b5fd740 1 -- start start 2024-03-23T13:20:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.448+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ad7f0 0x558e8f9adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 0x558e8f9b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa122d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ad7f0 0x558e8f9adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa1137fe640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa10c0a20d0 0x7fa10c0a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa122d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ad7f0 0x558e8f9adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:44678/0 (socket says 172.21.15.7:44678) 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa12259c640 1 --2- 172.21.15.7:0/2164536178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 0x558e8f9b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa122d9d640 1 -- 172.21.15.7:0/2164536178 learned_addr learned my addr 172.21.15.7:0/2164536178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/2164536178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9b3350 0x558e8f9b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa12b5fd740 1 -- 172.21.15.7:0/2164536178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558e8f9691b0 con 0x558e8f9b3350 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa121d9b640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa10c0a20d0 0x7fa10c0a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa1137fe640 1 -- 172.21.15.7:0/2937550523 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa11c0ef9e0 con 0x7fa124109820 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa12b5fd740 1 -- 172.21.15.7:0/2164536178 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558e8f91bf80 con 0x558e8f9ae1e0 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.449+0000 7fa12b5fd740 1 -- 172.21.15.7:0/2164536178 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558e8f9194b0 con 0x558e8f9ad7f0 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa12259c640 1 -- 172.21.15.7:0/2164536178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ad7f0 msgr2=0x558e8f9adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa12259c640 1 --2- 172.21.15.7:0/2164536178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ad7f0 0x558e8f9adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa12259c640 1 -- 172.21.15.7:0/2164536178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9b3350 msgr2=0x558e8f9b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa12259c640 1 --2- 172.21.15.7:0/2164536178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9b3350 0x558e8f9b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa12259c640 1 -- 172.21.15.7:0/2164536178 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e8f8aa810 con 0x558e8f9ae1e0 2024-03-23T13:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa121d9b640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa10c0a20d0 0x7fa10c0a4590 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa118009fd0 tx=0x7fa118009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa122d9d640 1 --2- 172.21.15.7:0/2164536178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ad7f0 0x558e8f9adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa12259c640 1 --2- 172.21.15.7:0/2164536178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 0x558e8f9b2ce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0bc3c0 tx=0x7fa11c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dabd9b4ec14bc2b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa110ff9640 1 -- 172.21.15.7:0/2164536178 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0bc200 con 0x558e8f9ae1e0 2024-03-23T13:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa110ff9640 1 -- 172.21.15.7:0/2164536178 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11c0bc020 con 0x558e8f9ae1e0 2024-03-23T13:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.450+0000 7fa110ff9640 1 -- 172.21.15.7:0/2164536178 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c003db0 con 0x558e8f9ae1e0 2024-03-23T13:20:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.451+0000 7fa12b5fd740 1 -- 172.21.15.7:0/2164536178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 msgr2=0x558e8f9b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.451+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/2164536178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 0x558e8f9b2ce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0bc3c0 tx=0x7fa11c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:20:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.451+0000 7fa12b5fd740 1 -- 172.21.15.7:0/2164536178 shutdown_connections 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.451+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/2164536178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9b3350 0x558e8f9b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.451+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/2164536178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 0x558e8f9b2ce0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.451+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/2164536178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ad7f0 0x558e8f9adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.451+0000 7fa12b5fd740 1 -- 172.21.15.7:0/2164536178 >> 172.21.15.7:0/2164536178 conn(0x558e8f9a8d60 msgr2=0x558e8f9a9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.451+0000 7fa12b5fd740 1 -- 172.21.15.7:0/2164536178 shutdown_connections 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.451+0000 7fa12b5fd740 1 -- 172.21.15.7:0/2164536178 wait complete. 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.452+0000 7fa12b5fd740 1 -- start start 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.452+0000 7fa12b5fd740 1 Processor -- start 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.452+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 0x558e8f9b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.452+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 0x558e8f9b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.452+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9b3350 0x558e8f9beac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.452+0000 7fa12259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 0x558e8f9b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.452+0000 7fa12259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 0x558e8f9b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:55896/0 (socket says 172.21.15.7:55896) 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.452+0000 7fa12259c640 1 -- 172.21.15.7:0/3229993656 learned_addr learned my addr 172.21.15.7:0/3229993656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.452+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558e8f914c90 con 0x558e8f9b3350 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.452+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558e8f8b9c50 con 0x558e8f9ad7f0 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558e8f5e8250 con 0x558e8f9ae1e0 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa121d9b640 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 0x558e8f9b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa122d9d640 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9b3350 0x558e8f9beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa12b5fd740 10 client.0 _ll_register_callbacks cb 0x558e8f908290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa121d9b640 1 -- 172.21.15.7:0/3229993656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 msgr2=0x558e8f9b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa121d9b640 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 0x558e8f9b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa121d9b640 1 -- 172.21.15.7:0/3229993656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9b3350 msgr2=0x558e8f9beac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa121d9b640 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9b3350 0x558e8f9beac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa121d9b640 1 -- 172.21.15.7:0/3229993656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12419dd00 con 0x558e8f9ae1e0 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa121d9b640 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 0x558e8f9b7af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1241a3630 tx=0x7fa118004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cacca167d5c24093 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa109ffb640 1 -- 172.21.15.7:0/3229993656 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118004d20 con 0x558e8f9ae1e0 2024-03-23T13:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa109ffb640 1 -- 172.21.15.7:0/3229993656 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa118004820 con 0x558e8f9ae1e0 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e8f939790 con 0x558e8f9ae1e0 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa109ffb640 1 -- 172.21.15.7:0/3229993656 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa118004e80 con 0x558e8f9ae1e0 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa12b5fd740 10 client.25255 fetch_fsmap learned FSMap version 27 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.453+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e8f913310 con 0x558e8f9ae1e0 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.454+0000 7fa109ffb640 1 -- 172.21.15.7:0/3229993656 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11802c070 con 0x558e8f9ae1e0 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.454+0000 7fa109ffb640 1 -- 172.21.15.7:0/3229993656 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa11802f070 con 0x558e8f9ae1e0 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.454+0000 7fa12b5fd740 10 client.25255 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.454+0000 7fa12b5fd740 10 client.25255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.454+0000 7fa12b5fd740 20 client.25255 populate_metadata read hostname 'smithi007' 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.454+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558e8f9c3540 0x558e8f9c5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.454+0000 7fa12b5fd740 4 client.25255 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.454+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e8f939790 con 0x558e8f9c3540 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.455+0000 7fa12259c640 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558e8f9c3540 0x558e8f9c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.455+0000 7fa109ffb640 10 client.25255 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.455+0000 7fa12259c640 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558e8f9c3540 0x558e8f9c5970 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4412599513c9e7 server_cookie=848870b6a1204bbe in_seq=0 out_seq=0 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa109ffb640 1 -- 172.21.15.7:0/3229993656 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558e8f939790 con 0x558e8f9c3540 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa109ffb640 10 client.25255 handle_command_reply: tid=0 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 client.25255 shutdown 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 msgr2=0x558e8f9b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 0x558e8f9b7af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1241a3630 tx=0x7fa118004ab0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 shutdown_connections 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558e8f9c3540 0x558e8f9c5970 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9b3350 0x558e8f9beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa109ffb640 0 client.25255 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 0x558e8f9b7af0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/3229993656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 0x558e8f9b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 >> 172.21.15.7:0/3229993656 conn(0x558e8f9a8d60 msgr2=0x558e8f9a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 shutdown_connections 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3229993656 wait complete. 2024-03-23T13:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.456+0000 7fa12b5fd740 20 client.25255 trim_cache size 0 max 16384 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12b5fd740 1 Processor -- start 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12b5fd740 1 -- start start 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9a8d60 0x558e8f9b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 0x558e8f9b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 0x558e8f9bea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12b5fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558e8f89a660 con 0x558e8f9a8d60 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9a8d60 0x558e8f9b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9a8d60 0x558e8f9b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37004/0 (socket says 172.21.15.7:37004) 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12259c640 1 -- 172.21.15.7:0/3952564050 learned_addr learned my addr 172.21.15.7:0/3952564050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa121d9b640 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 0x558e8f9b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa122d9d640 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 0x558e8f9bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3952564050 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558e8f9c1990 con 0x558e8f9ad7f0 2024-03-23T13:20:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.485+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3952564050 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558e8f9397c0 con 0x558e8f9ae1e0 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa12259c640 1 -- 172.21.15.7:0/3952564050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 msgr2=0x558e8f9bea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa12259c640 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 0x558e8f9bea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa12259c640 1 -- 172.21.15.7:0/3952564050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 msgr2=0x558e8f9b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa12259c640 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 0x558e8f9b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa12259c640 1 -- 172.21.15.7:0/3952564050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e8f8af810 con 0x558e8f9a8d60 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa121d9b640 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 0x558e8f9b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa122d9d640 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 0x558e8f9bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa12259c640 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9a8d60 0x558e8f9b75b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa11c005ab0 tx=0x7fa11c0a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae43f0bb23d606ff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa10bfff640 1 -- 172.21.15.7:0/3952564050 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0056f0 con 0x558e8f9a8d60 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa10bfff640 1 -- 172.21.15.7:0/3952564050 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11c0bcdc0 con 0x558e8f9a8d60 2024-03-23T13:20:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.486+0000 7fa10bfff640 1 -- 172.21.15.7:0/3952564050 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0ba9c0 con 0x558e8f9a8d60 2024-03-23T13:20:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.487+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3952564050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9a8d60 msgr2=0x558e8f9b75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.487+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9a8d60 0x558e8f9b75b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa11c005ab0 tx=0x7fa11c0a7040 comp rx=0 tx=0).stop 2024-03-23T13:20:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.487+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3952564050 shutdown_connections 2024-03-23T13:20:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.487+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9ae1e0 0x558e8f9bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.487+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ad7f0 0x558e8f9b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.487+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/3952564050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9a8d60 0x558e8f9b75b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.487+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3952564050 >> 172.21.15.7:0/3952564050 conn(0x558e8f9b2e90 msgr2=0x558e8f9a9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.487+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3952564050 shutdown_connections 2024-03-23T13:20:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.487+0000 7fa12b5fd740 1 -- 172.21.15.7:0/3952564050 wait complete. 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.487+0000 7fa12b5fd740 1 -- start start 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12b5fd740 1 Processor -- start 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9a8d60 0x558e8f9bc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9ad7f0 0x558e8f9bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12b5fd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 0x558e8f9bd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12b5fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558e8f9bda40 con 0x558e8f9ad7f0 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12b5fd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558e8f9bdbb0 con 0x558e8f9ae1e0 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12b5fd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558e8f9bdd70 con 0x558e8f9a8d60 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12b5fd740 10 client.0 _ll_register_callbacks cb 0x7fa118011e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12259c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9a8d60 0x558e8f9bc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12259c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9a8d60 0x558e8f9bc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:44710/0 (socket says 172.21.15.7:44710) 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12259c640 1 -- 172.21.15.7:0/493791228 learned_addr learned my addr 172.21.15.7:0/493791228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa121d9b640 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9ad7f0 0x558e8f9bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa122d9d640 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 0x558e8f9bd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12259c640 1 -- 172.21.15.7:0/493791228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 msgr2=0x558e8f9bd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12259c640 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 0x558e8f9bd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12259c640 1 -- 172.21.15.7:0/493791228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9ad7f0 msgr2=0x558e8f9bc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12259c640 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9ad7f0 0x558e8f9bc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa12259c640 1 -- 172.21.15.7:0/493791228 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e8f913310 con 0x558e8f9a8d60 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.488+0000 7fa121d9b640 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9ad7f0 0x558e8f9bc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa12259c640 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9a8d60 0x558e8f9bc020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0b8b20 tx=0x7fa11c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6080d645d316686 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa109ffb640 1 -- 172.21.15.7:0/493791228 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0056f0 con 0x558e8f9a8d60 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa109ffb640 1 -- 172.21.15.7:0/493791228 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa11c0c4e60 con 0x558e8f9a8d60 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa12b5fd740 1 -- 172.21.15.7:0/493791228 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa11c0bc020 con 0x558e8f9a8d60 2024-03-23T13:20:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa109ffb640 1 -- 172.21.15.7:0/493791228 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa11c0a0020 con 0x558e8f9a8d60 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa12b5fd740 10 client.25261 fetch_fsmap learned FSMap version 27 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa12b5fd740 1 -- 172.21.15.7:0/493791228 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x7fa114004120 con 0x558e8f9a8d60 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa109ffb640 1 -- 172.21.15.7:0/493791228 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11c0b8d30 con 0x558e8f9a8d60 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa109ffb640 1 -- 172.21.15.7:0/493791228 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa11c03c070 con 0x558e8f9a8d60 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa12b5fd740 10 client.25261 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa12b5fd740 10 client.25261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.489+0000 7fa12b5fd740 20 client.25261 populate_metadata read hostname 'smithi007' 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.490+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558e8f9b3350 0x558e8fa2bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.490+0000 7fa12b5fd740 4 client.25261 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.490+0000 7fa12b5fd740 1 -- 172.21.15.7:0/493791228 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa11c0bc020 con 0x558e8f9b3350 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.490+0000 7fa121d9b640 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558e8f9b3350 0x558e8fa2bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.490+0000 7fa109ffb640 10 client.25261 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.490+0000 7fa121d9b640 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558e8f9b3350 0x558e8fa2bb40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124a8eb07091e4ef server_cookie=bc7f0839fdf0dd4e in_seq=0 out_seq=0 2024-03-23T13:20:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa109ffb640 1 -- 172.21.15.7:0/493791228 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa11c0bc020 con 0x558e8f9b3350 2024-03-23T13:20:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa109ffb640 10 client.25261 handle_command_reply: tid=0 2024-03-23T13:20:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 client.25261 shutdown 2024-03-23T13:20:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 -- 172.21.15.7:0/493791228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9a8d60 msgr2=0x558e8f9bc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9a8d60 0x558e8f9bc020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0b8b20 tx=0x7fa11c000bc0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 -- 172.21.15.7:0/493791228 shutdown_connections 2024-03-23T13:20:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558e8f9b3350 0x558e8fa2bb40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558e8f9ae1e0 0x558e8f9bd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e8f9ad7f0 0x558e8f9bc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa109ffb640 0 client.25261 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 --2- 172.21.15.7:0/493791228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e8f9a8d60 0x558e8f9bc020 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 -- 172.21.15.7:0/493791228 >> 172.21.15.7:0/493791228 conn(0x558e8f9b2e90 msgr2=0x558e8f9a9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 -- 172.21.15.7:0/493791228 shutdown_connections 2024-03-23T13:20:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 1 -- 172.21.15.7:0/493791228 wait complete. 2024-03-23T13:20:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.497+0000 7fa12b5fd740 20 client.25261 trim_cache size 0 max 16384 2024-03-23T13:20:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-23T13:20:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-23T13:20:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-23T13:20:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-23T13:20:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 -- 172.21.15.7:0/2937550523 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa10c0a20d0 msgr2=0x7fa10c0a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa10c0a20d0 0x7fa10c0a4590 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa118009fd0 tx=0x7fa118009500 comp rx=0 tx=0).stop 2024-03-23T13:20:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 -- 172.21.15.7:0/2937550523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124109820 msgr2=0x7fa1241a2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124109820 0x7fa1241a2120 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006010 tx=0x7fa11c005e80 comp rx=0 tx=0).stop 2024-03-23T13:20:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 -- 172.21.15.7:0/2937550523 shutdown_connections 2024-03-23T13:20:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fa10c0a20d0 0x7fa10c0a4590 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12410ead0 0x7fa1241a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12410c150 0x7fa1241a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 --2- 172.21.15.7:0/2937550523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124109820 0x7fa1241a2120 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 -- 172.21.15.7:0/2937550523 >> 172.21.15.7:0/2937550523 conn(0x7fa12406e750 msgr2=0x7fa12410b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 -- 172.21.15.7:0/2937550523 shutdown_connections 2024-03-23T13:20:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:20.498+0000 7fa128d81640 1 -- 172.21.15.7:0/2937550523 wait complete. 2024-03-23T13:20:21.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:21 smithi155 ceph-mon[36157]: pgmap v366: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:20:21.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:21 smithi007 ceph-mon[32184]: pgmap v366: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:20:21.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:21 smithi130 ceph-mon[36225]: pgmap v366: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T13:20:23.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:23 smithi155 ceph-mon[36157]: pgmap v367: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-23T13:20:23.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:23 smithi007 ceph-mon[32184]: pgmap v367: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-23T13:20:23.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:23 smithi130 ceph-mon[36225]: pgmap v367: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-23T13:20:24.224 INFO:tasks.workunit.client.0.smithi007.stderr:Note: switching to 'd39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b'. 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr:state without impacting any branches by switching back to a branch. 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr: git switch -c 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr:Or undo this operation with: 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:20:24.225 INFO:tasks.workunit.client.0.smithi007.stderr: git switch - 2024-03-23T13:20:24.226 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:20:24.226 INFO:tasks.workunit.client.0.smithi007.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-23T13:20:24.226 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:20:24.226 INFO:tasks.workunit.client.0.smithi007.stderr:HEAD is now at d39ab7bb1f1 qa/suites/fs/workload: enable snap_schedule early 2024-03-23T13:20:24.230 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-23T13:20:24.295 INFO:tasks.workunit.client.0.smithi007.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-23T13:20:24.298 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-23T13:20:24.298 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-23T13:20:25.063 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-23T13:20:25.129 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-23T13:20:25.169 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-23T13:20:25.171 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-23T13:20:25.171 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-23T13:20:25.210 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-23T13:20:25.213 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:20:25.213 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-23T13:20:25.268 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-03-23T13:20:25.269 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-03-23T13:20:25.269 DEBUG:teuthology.orchestra.run.smithi007:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-03-23T13:20:25.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:25 smithi155 ceph-mon[36157]: pgmap v368: 97 pgs: 97 active+clean; 664 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-23T13:20:25.328 INFO:tasks.workunit.client.0.smithi007.stderr:+ set -e 2024-03-23T13:20:25.328 INFO:tasks.workunit.client.0.smithi007.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-03-23T13:20:25.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:25 smithi007 ceph-mon[32184]: pgmap v368: 97 pgs: 97 active+clean; 664 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-23T13:20:25.330 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into 'xfstests-dev'... 2024-03-23T13:20:25.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:25 smithi130 ceph-mon[36225]: pgmap v368: 97 pgs: 97 active+clean; 664 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-23T13:20:27.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:27 smithi155 ceph-mon[36157]: pgmap v369: 97 pgs: 97 active+clean; 664 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-23T13:20:27.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:27 smithi007 ceph-mon[32184]: pgmap v369: 97 pgs: 97 active+clean; 664 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-23T13:20:27.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:27 smithi130 ceph-mon[36225]: pgmap v369: 97 pgs: 97 active+clean; 664 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-23T13:20:29.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:29 smithi155 ceph-mon[36157]: pgmap v370: 97 pgs: 97 active+clean; 826 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-03-23T13:20:29.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:29 smithi007 ceph-mon[32184]: pgmap v370: 97 pgs: 97 active+clean; 826 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-03-23T13:20:29.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:29 smithi130 ceph-mon[36225]: pgmap v370: 97 pgs: 97 active+clean; 826 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-03-23T13:20:31.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:31 smithi155 ceph-mon[36157]: pgmap v371: 97 pgs: 97 active+clean; 826 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-03-23T13:20:31.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:31 smithi007 ceph-mon[32184]: pgmap v371: 97 pgs: 97 active+clean; 826 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-03-23T13:20:31.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:31 smithi130 ceph-mon[36225]: pgmap v371: 97 pgs: 97 active+clean; 826 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-03-23T13:20:33.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:33 smithi155 ceph-mon[36157]: pgmap v372: 97 pgs: 97 active+clean; 826 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-23T13:20:33.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:33 smithi007 ceph-mon[32184]: pgmap v372: 97 pgs: 97 active+clean; 826 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-23T13:20:33.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:33 smithi130 ceph-mon[36225]: pgmap v372: 97 pgs: 97 active+clean; 826 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-03-23T13:20:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.597+0000 7fb98423b640 1 -- 172.21.15.7:0/2458658913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb97c101870 msgr2=0x7fb97c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.597+0000 7fb98423b640 1 --2- 172.21.15.7:0/2458658913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb97c101870 0x7fb97c101cf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009900 tx=0x7fb96c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:20:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 -- 172.21.15.7:0/2458658913 shutdown_connections 2024-03-23T13:20:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 --2- 172.21.15.7:0/2458658913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb97c102230 0x7fb97c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 --2- 172.21.15.7:0/2458658913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb97c101870 0x7fb97c101cf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 --2- 172.21.15.7:0/2458658913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb97c100680 0x7fb97c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 -- 172.21.15.7:0/2458658913 >> 172.21.15.7:0/2458658913 conn(0x7fb97c0fbbf0 msgr2=0x7fb97c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 -- 172.21.15.7:0/2458658913 shutdown_connections 2024-03-23T13:20:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 -- 172.21.15.7:0/2458658913 wait complete. 2024-03-23T13:20:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 Processor -- start 2024-03-23T13:20:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 -- start start 2024-03-23T13:20:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb97c100680 0x7fb97c1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb97c101870 0x7fb97c196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb97c102230 0x7fb97c19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c19e1b0 con 0x7fb97c102230 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c19e320 con 0x7fb97c101870 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.598+0000 7fb98423b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c19e4e0 con 0x7fb97c100680 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb9827d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb97c102230 0x7fb97c19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb981fd7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb97c100680 0x7fb97c1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb9827d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb97c102230 0x7fb97c19dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42588/0 (socket says 172.21.15.7:42588) 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb9827d8640 1 -- 172.21.15.7:0/1843054652 learned_addr learned my addr 172.21.15.7:0/1843054652 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb9817d6640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb97c101870 0x7fb97c196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb981fd7640 1 -- 172.21.15.7:0/1843054652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb97c101870 msgr2=0x7fb97c196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb981fd7640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb97c101870 0x7fb97c196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb981fd7640 1 -- 172.21.15.7:0/1843054652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb97c102230 msgr2=0x7fb97c19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb981fd7640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb97c102230 0x7fb97c19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb981fd7640 1 -- 172.21.15.7:0/1843054652 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c009560 con 0x7fb97c100680 2024-03-23T13:20:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb9827d8640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb97c102230 0x7fb97c19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb9817d6640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb97c101870 0x7fb97c196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb981fd7640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb97c100680 0x7fb97c1966c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb97800e9c0 tx=0x7fb97800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb972ffd640 1 -- 172.21.15.7:0/1843054652 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97800ccc0 con 0x7fb97c100680 2024-03-23T13:20:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb98423b640 1 -- 172.21.15.7:0/1843054652 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c19e7c0 con 0x7fb97c100680 2024-03-23T13:20:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.599+0000 7fb972ffd640 1 -- 172.21.15.7:0/1843054652 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97800ce20 con 0x7fb97c100680 2024-03-23T13:20:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.600+0000 7fb972ffd640 1 -- 172.21.15.7:0/1843054652 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978010430 con 0x7fb97c100680 2024-03-23T13:20:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.600+0000 7fb98423b640 1 -- 172.21.15.7:0/1843054652 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c19eca0 con 0x7fb97c100680 2024-03-23T13:20:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb986e17740 1 Processor -- start 2024-03-23T13:20:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb972ffd640 1 -- 172.21.15.7:0/1843054652 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fb978003f60 con 0x7fb97c100680 2024-03-23T13:20:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb986e17740 1 -- start start 2024-03-23T13:20:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb986e17740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79af7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb986e17740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 0x55cf79afcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb986e17740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79afd560 0x55cf79aff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb986e17740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cf79ab11b0 con 0x55cf79af7a00 2024-03-23T13:20:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb986e17740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cf79a63f80 con 0x55cf79af83f0 2024-03-23T13:20:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb986e17740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf79a614b0 con 0x55cf79afd560 2024-03-23T13:20:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb9827d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79af7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb9827d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79af7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42600/0 (socket says 172.21.15.7:42600) 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb9827d8640 1 -- 172.21.15.7:0/2527544614 learned_addr learned my addr 172.21.15.7:0/2527544614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb981fd7640 1 --2- 172.21.15.7:0/2527544614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 0x55cf79afcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb9817d6640 1 --2- 172.21.15.7:0/2527544614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79afd560 0x55cf79aff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb972ffd640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9680a2160 0x7fb9680a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb9827d8640 1 -- 172.21.15.7:0/2527544614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79afd560 msgr2=0x55cf79aff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb9827d8640 1 --2- 172.21.15.7:0/2527544614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79afd560 0x55cf79aff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb9827d8640 1 -- 172.21.15.7:0/2527544614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 msgr2=0x55cf79afcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb9827d8640 1 --2- 172.21.15.7:0/2527544614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 0x55cf79afcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.603+0000 7fb9827d8640 1 -- 172.21.15.7:0/2527544614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf799f2810 con 0x55cf79af7a00 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.604+0000 7fb972ffd640 1 -- 172.21.15.7:0/1843054652 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb978014070 con 0x7fb97c100680 2024-03-23T13:20:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.604+0000 7fb9827d8640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9680a2160 0x7fb9680a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.604+0000 7fb9827d8640 1 --2- 172.21.15.7:0/2527544614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79af7e20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb974011850 tx=0x7fb974011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6124ce00ea0f7988 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.604+0000 7fb967fff640 1 -- 172.21.15.7:0/2527544614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97400fba0 con 0x55cf79af7a00 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.604+0000 7fb967fff640 1 -- 172.21.15.7:0/2527544614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97400fea0 con 0x55cf79af7a00 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.604+0000 7fb967fff640 1 -- 172.21.15.7:0/2527544614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97401ba60 con 0x55cf79af7a00 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 -- 172.21.15.7:0/2527544614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 msgr2=0x55cf79af7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb9827d8640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9680a2160 0x7fb9680a4620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb9740114d0 tx=0x7fb97400e780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 --2- 172.21.15.7:0/2527544614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79af7e20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb974011850 tx=0x7fb974011d10 comp rx=0 tx=0).stop 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 -- 172.21.15.7:0/2527544614 shutdown_connections 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 --2- 172.21.15.7:0/2527544614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79afd560 0x55cf79aff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 --2- 172.21.15.7:0/2527544614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 0x55cf79afcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 --2- 172.21.15.7:0/2527544614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79af7e20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 -- 172.21.15.7:0/2527544614 >> 172.21.15.7:0/2527544614 conn(0x55cf79af0d60 msgr2=0x55cf79af1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 -- 172.21.15.7:0/2527544614 shutdown_connections 2024-03-23T13:20:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 -- 172.21.15.7:0/2527544614 wait complete. 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 -- start start 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.605+0000 7fb986e17740 1 Processor -- start 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb986e17740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 0x55cf79b01890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb986e17740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b01dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb986e17740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79afd560 0x55cf79b08e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb986e17740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cf79a5cc90 con 0x55cf79af83f0 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb986e17740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cf79a01c50 con 0x55cf79afd560 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb986e17740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf79730250 con 0x55cf79af7a00 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb986e17740 10 client.0 _ll_register_callbacks cb 0x55cf79a50290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb9817d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b01dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb981fd7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 0x55cf79b01890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb9817d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b01dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42602/0 (socket says 172.21.15.7:42602) 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb9817d6640 1 -- 172.21.15.7:0/1863452468 learned_addr learned my addr 172.21.15.7:0/1863452468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb9827d8640 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79afd560 0x55cf79b08e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb981fd7640 1 -- 172.21.15.7:0/1863452468 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79afd560 msgr2=0x55cf79b08e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb981fd7640 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79afd560 0x55cf79b08e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb981fd7640 1 -- 172.21.15.7:0/1863452468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 msgr2=0x55cf79b01dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb981fd7640 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b01dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb981fd7640 1 -- 172.21.15.7:0/1863452468 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb974011500 con 0x55cf79af7a00 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb9817d6640 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b01dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb981fd7640 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 0x55cf79b01890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9780cbc50 tx=0x7fb9780970d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=303c51f1cf5f3abb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.606+0000 7fb9657fa640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb9657fa640 1 -- 172.21.15.7:0/1863452468 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97809a790 con 0x55cf79af7a00 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb986e17740 1 -- 172.21.15.7:0/1863452468 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cf79a81790 con 0x55cf79af7a00 2024-03-23T13:20:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb9657fa640 1 -- 172.21.15.7:0/1863452468 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb97809b530 con 0x55cf79af7a00 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb9657fa640 1 -- 172.21.15.7:0/1863452468 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb97809ae30 con 0x55cf79af7a00 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb986e17740 10 client.25273 fetch_fsmap learned FSMap version 27 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb986e17740 1 -- 172.21.15.7:0/1863452468 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf79a5b310 con 0x55cf79af7a00 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb9657fa640 1 -- 172.21.15.7:0/1863452468 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cf79a81790 con 0x55cf79af7a00 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb9657fa640 1 -- 172.21.15.7:0/1863452468 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb97801f060 con 0x55cf79af7a00 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb986e17740 10 client.25273 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb986e17740 10 client.25273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb986e17740 20 client.25273 populate_metadata read hostname 'smithi007' 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb986e17740 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55cf79b0d810 0x55cf79b0fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb986e17740 4 client.25273 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb986e17740 1 -- 172.21.15.7:0/1863452468 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf7975c600 con 0x55cf79b0d810 2024-03-23T13:20:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.607+0000 7fb9817d6640 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55cf79b0d810 0x55cf79b0fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.608+0000 7fb9657fa640 10 client.25273 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.608+0000 7fb9817d6640 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55cf79b0d810 0x55cf79b0fc40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3181b385ae930da server_cookie=53f766e2df2b4358 in_seq=0 out_seq=0 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.608+0000 7fb9657fa640 1 -- 172.21.15.7:0/1863452468 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cf7975c600 con 0x55cf79b0d810 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.608+0000 7fb9657fa640 10 client.25273 handle_command_reply: tid=0 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 client.25273 shutdown 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 -- 172.21.15.7:0/1863452468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 msgr2=0x55cf79b01890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 0x55cf79b01890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9780cbc50 tx=0x7fb9780970d0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 -- 172.21.15.7:0/1863452468 shutdown_connections 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55cf79b0d810 0x55cf79b0fc40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79afd560 0x55cf79b08e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b01dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 --2- 172.21.15.7:0/1863452468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 0x55cf79b01890 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb9657fa640 0 client.25273 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 -- 172.21.15.7:0/1863452468 >> 172.21.15.7:0/1863452468 conn(0x55cf79af0d60 msgr2=0x55cf79aefea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 -- 172.21.15.7:0/1863452468 shutdown_connections 2024-03-23T13:20:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 1 -- 172.21.15.7:0/1863452468 wait complete. 2024-03-23T13:20:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.609+0000 7fb986e17740 20 client.25273 trim_cache size 0 max 16384 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb986e17740 1 Processor -- start 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb986e17740 1 -- start start 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb986e17740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79b15520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb986e17740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 0x55cf79b08dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb986e17740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79afd560 0x55cf79af3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb986e17740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cf799e2660 con 0x55cf79af7a00 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb986e17740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cf79a81790 con 0x55cf79af83f0 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb986e17740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf79af90c0 con 0x55cf79afd560 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb981fd7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79b15520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb981fd7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79b15520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42612/0 (socket says 172.21.15.7:42612) 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb981fd7640 1 -- 172.21.15.7:0/4079779423 learned_addr learned my addr 172.21.15.7:0/4079779423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb9827d8640 1 --2- 172.21.15.7:0/4079779423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79afd560 0x55cf79af3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb9817d6640 1 --2- 172.21.15.7:0/4079779423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 0x55cf79b08dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb981fd7640 1 -- 172.21.15.7:0/4079779423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79afd560 msgr2=0x55cf79af3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb981fd7640 1 --2- 172.21.15.7:0/4079779423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79afd560 0x55cf79af3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb981fd7640 1 -- 172.21.15.7:0/4079779423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 msgr2=0x55cf79b08dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb981fd7640 1 --2- 172.21.15.7:0/4079779423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 0x55cf79b08dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb981fd7640 1 -- 172.21.15.7:0/4079779423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf799f7810 con 0x55cf79af7a00 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.624+0000 7fb9817d6640 1 --2- 172.21.15.7:0/4079779423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 0x55cf79b08dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb981fd7640 1 --2- 172.21.15.7:0/4079779423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79b15520 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9780cbf70 tx=0x7fb97800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72abb049010c9ff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb9677fe640 1 -- 172.21.15.7:0/4079779423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780d5070 con 0x55cf79af7a00 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb9677fe640 1 -- 172.21.15.7:0/4079779423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9780a35d0 con 0x55cf79af7a00 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb9677fe640 1 -- 172.21.15.7:0/4079779423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780a3140 con 0x55cf79af7a00 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb986e17740 1 -- 172.21.15.7:0/4079779423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 msgr2=0x55cf79b15520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb986e17740 1 --2- 172.21.15.7:0/4079779423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79b15520 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9780cbf70 tx=0x7fb97800cf90 comp rx=0 tx=0).stop 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb986e17740 1 -- 172.21.15.7:0/4079779423 shutdown_connections 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb986e17740 1 --2- 172.21.15.7:0/4079779423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79afd560 0x55cf79af3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb986e17740 1 --2- 172.21.15.7:0/4079779423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79af83f0 0x55cf79b08dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb986e17740 1 --2- 172.21.15.7:0/4079779423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af7a00 0x55cf79b15520 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb986e17740 1 -- 172.21.15.7:0/4079779423 >> 172.21.15.7:0/4079779423 conn(0x55cf79af0d60 msgr2=0x55cf79af1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb986e17740 1 -- 172.21.15.7:0/4079779423 shutdown_connections 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.625+0000 7fb986e17740 1 -- 172.21.15.7:0/4079779423 wait complete. 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb986e17740 1 -- start start 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb986e17740 1 Processor -- start 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb986e17740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 0x55cf79b05f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb986e17740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb986e17740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79afd560 0x55cf79b07460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb986e17740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cf79b079a0 con 0x55cf79af83f0 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb986e17740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cf79b07b10 con 0x55cf79afd560 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb986e17740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf79b07cd0 con 0x55cf79af7a00 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb986e17740 10 client.0 _ll_register_callbacks cb 0x7fb978003880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb9817d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb9817d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b064a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42616/0 (socket says 172.21.15.7:42616) 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb9817d6640 1 -- 172.21.15.7:0/190961146 learned_addr learned my addr 172.21.15.7:0/190961146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb981fd7640 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 0x55cf79b05f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.626+0000 7fb9827d8640 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79afd560 0x55cf79b07460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9817d6640 1 -- 172.21.15.7:0/190961146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 msgr2=0x55cf79b05f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9817d6640 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 0x55cf79b05f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9817d6640 1 -- 172.21.15.7:0/190961146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79afd560 msgr2=0x55cf79b07460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9817d6640 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79afd560 0x55cf79b07460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9817d6640 1 -- 172.21.15.7:0/190961146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf79a5b310 con 0x55cf79af83f0 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9817d6640 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b064a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb96c036040 tx=0x7fb96c02cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb13aea8664dda5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9657fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9657fa640 1 -- 172.21.15.7:0/190961146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c04ade0 con 0x55cf79af83f0 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9657fa640 1 -- 172.21.15.7:0/190961146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb96c046040 con 0x55cf79af83f0 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb986e17740 1 -- 172.21.15.7:0/190961146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb97400fea0 con 0x55cf79af83f0 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9657fa640 1 -- 172.21.15.7:0/190961146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb96c0040c0 con 0x55cf79af83f0 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb986e17740 10 client.15426 fetch_fsmap learned FSMap version 27 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb986e17740 1 -- 172.21.15.7:0/190961146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf79a1d5a0 con 0x55cf79af83f0 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9657fa640 1 -- 172.21.15.7:0/190961146 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97400fea0 con 0x55cf79af83f0 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb9657fa640 1 -- 172.21.15.7:0/190961146 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb96c03c040 con 0x55cf79af83f0 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.627+0000 7fb986e17740 10 client.15426 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T13:20:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.628+0000 7fb986e17740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:20:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.628+0000 7fb986e17740 20 client.15426 populate_metadata read hostname 'smithi007' 2024-03-23T13:20:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.628+0000 7fb986e17740 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55cf79b74060 0x55cf79b76450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.628+0000 7fb986e17740 4 client.15426 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T13:20:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.628+0000 7fb986e17740 1 -- 172.21.15.7:0/190961146 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf797ad1b0 con 0x55cf79b74060 2024-03-23T13:20:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.628+0000 7fb981fd7640 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55cf79b74060 0x55cf79b76450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.628+0000 7fb981fd7640 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55cf79b74060 0x55cf79b76450 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57fe1519743f86ba server_cookie=f0c0ff02da4c08f5 in_seq=0 out_seq=0 2024-03-23T13:20:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.628+0000 7fb9657fa640 10 client.15426 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb9657fa640 1 -- 172.21.15.7:0/190961146 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf797ad1b0 con 0x55cf79b74060 2024-03-23T13:20:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb9657fa640 10 client.15426 handle_command_reply: tid=0 2024-03-23T13:20:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 client.15426 shutdown 2024-03-23T13:20:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 -- 172.21.15.7:0/190961146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 msgr2=0x55cf79b064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b064a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb96c036040 tx=0x7fb96c02cc90 comp rx=0 tx=0).stop 2024-03-23T13:20:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 -- 172.21.15.7:0/190961146 shutdown_connections 2024-03-23T13:20:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55cf79b74060 0x55cf79b76450 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf79afd560 0x55cf79b07460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb9657fa640 0 client.15426 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf79af83f0 0x55cf79b064a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 --2- 172.21.15.7:0/190961146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf79af7a00 0x55cf79b05f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 -- 172.21.15.7:0/190961146 >> 172.21.15.7:0/190961146 conn(0x55cf79af0d60 msgr2=0x55cf79af3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 -- 172.21.15.7:0/190961146 shutdown_connections 2024-03-23T13:20:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 1 -- 172.21.15.7:0/190961146 wait complete. 2024-03-23T13:20:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.859+0000 7fb986e17740 20 client.15426 trim_cache size 0 max 16384 2024-03-23T13:20:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-23T13:20:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-23T13:20:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-23T13:20:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-23T13:20:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 -- 172.21.15.7:0/1843054652 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9680a2160 msgr2=0x7fb9680a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9680a2160 0x7fb9680a4620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb9740114d0 tx=0x7fb97400e780 comp rx=0 tx=0).stop 2024-03-23T13:20:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 -- 172.21.15.7:0/1843054652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb97c100680 msgr2=0x7fb97c1966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb97c100680 0x7fb97c1966c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb97800e9c0 tx=0x7fb97800ee80 comp rx=0 tx=0).stop 2024-03-23T13:20:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 -- 172.21.15.7:0/1843054652 shutdown_connections 2024-03-23T13:20:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9680a2160 0x7fb9680a4620 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb97c102230 0x7fb97c19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb97c101870 0x7fb97c196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 --2- 172.21.15.7:0/1843054652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb97c100680 0x7fb97c1966c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 -- 172.21.15.7:0/1843054652 >> 172.21.15.7:0/1843054652 conn(0x7fb97c0fbbf0 msgr2=0x7fb97c104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 -- 172.21.15.7:0/1843054652 shutdown_connections 2024-03-23T13:20:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:33.860+0000 7fb98423b640 1 -- 172.21.15.7:0/1843054652 wait complete. 2024-03-23T13:20:35.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:35 smithi007 ceph-mon[32184]: pgmap v373: 97 pgs: 97 active+clean; 14 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 17 op/s 2024-03-23T13:20:35.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:35 smithi130 ceph-mon[36225]: pgmap v373: 97 pgs: 97 active+clean; 14 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 17 op/s 2024-03-23T13:20:35.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:35 smithi155 ceph-mon[36157]: pgmap v373: 97 pgs: 97 active+clean; 14 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 17 op/s 2024-03-23T13:20:37.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:37 smithi007 ceph-mon[32184]: pgmap v374: 97 pgs: 97 active+clean; 14 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-03-23T13:20:37.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:37 smithi130 ceph-mon[36225]: pgmap v374: 97 pgs: 97 active+clean; 14 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-03-23T13:20:37.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:37 smithi155 ceph-mon[36157]: pgmap v374: 97 pgs: 97 active+clean; 14 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-03-23T13:20:38.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:38 smithi007 ceph-mon[32184]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:20:38.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:38 smithi130 ceph-mon[36225]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:20:38.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:38 smithi155 ceph-mon[36157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:20:39.328 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:39 smithi007 ceph-mon[32184]: pgmap v375: 97 pgs: 97 active+clean; 16 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-23T13:20:39.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:39 smithi130 ceph-mon[36225]: pgmap v375: 97 pgs: 97 active+clean; 16 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-23T13:20:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:39 smithi155 ceph-mon[36157]: pgmap v375: 97 pgs: 97 active+clean; 16 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-23T13:20:41.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:41 smithi007 ceph-mon[32184]: pgmap v376: 97 pgs: 97 active+clean; 16 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-03-23T13:20:41.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:41 smithi130 ceph-mon[36225]: pgmap v376: 97 pgs: 97 active+clean; 16 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-03-23T13:20:41.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:41 smithi155 ceph-mon[36157]: pgmap v376: 97 pgs: 97 active+clean; 16 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-03-23T13:20:43.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:43 smithi007 ceph-mon[32184]: pgmap v377: 97 pgs: 97 active+clean; 18 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-03-23T13:20:43.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:43 smithi007 ceph-mon[32184]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:20:43.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:43 smithi130 ceph-mon[36225]: pgmap v377: 97 pgs: 97 active+clean; 18 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-03-23T13:20:43.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:43 smithi130 ceph-mon[36225]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:20:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:43 smithi155 ceph-mon[36157]: pgmap v377: 97 pgs: 97 active+clean; 18 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-03-23T13:20:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:43 smithi155 ceph-mon[36157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:20:45.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:45 smithi007 ceph-mon[32184]: pgmap v378: 97 pgs: 97 active+clean; 19 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-03-23T13:20:45.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:45 smithi130 ceph-mon[36225]: pgmap v378: 97 pgs: 97 active+clean; 19 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-03-23T13:20:45.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:45 smithi155 ceph-mon[36157]: pgmap v378: 97 pgs: 97 active+clean; 19 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-03-23T13:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.957+0000 7fd009d77640 1 -- 172.21.15.7:0/833196546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd004101760 msgr2=0x7fd004101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.957+0000 7fd009d77640 1 --2- 172.21.15.7:0/833196546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd004101760 0x7fd004101b80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009ca0 tx=0x7fcff402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.957+0000 7fd009d77640 1 -- 172.21.15.7:0/833196546 shutdown_connections 2024-03-23T13:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.957+0000 7fd009d77640 1 --2- 172.21.15.7:0/833196546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004103310 0x7fd004109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.957+0000 7fd009d77640 1 --2- 172.21.15.7:0/833196546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd004102950 0x7fd004102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.957+0000 7fd009d77640 1 --2- 172.21.15.7:0/833196546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd004101760 0x7fd004101b80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.957+0000 7fd009d77640 1 -- 172.21.15.7:0/833196546 >> 172.21.15.7:0/833196546 conn(0x7fd0040fccf0 msgr2=0x7fd0040ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.957+0000 7fd009d77640 1 -- 172.21.15.7:0/833196546 shutdown_connections 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.957+0000 7fd009d77640 1 -- 172.21.15.7:0/833196546 wait complete. 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.957+0000 7fd009d77640 1 Processor -- start 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd009d77640 1 -- start start 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd009d77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd004101760 0x7fd004197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd009d77640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004102950 0x7fd004197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd009d77640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd004103310 0x7fd00419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd009d77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd00419f260 con 0x7fd004101760 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd009d77640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd00419f3d0 con 0x7fd004102950 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd009d77640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd00419f590 con 0x7fd004103310 2024-03-23T13:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd0037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd004101760 0x7fd004197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd0037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd004101760 0x7fd004197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48090/0 (socket says 172.21.15.7:48090) 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd0037fe640 1 -- 172.21.15.7:0/2224933939 learned_addr learned my addr 172.21.15.7:0/2224933939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd002ffd640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004102950 0x7fd004197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd003fff640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd004103310 0x7fd00419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd0037fe640 1 -- 172.21.15.7:0/2224933939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd004103310 msgr2=0x7fd00419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd0037fe640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd004103310 0x7fd00419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd0037fe640 1 -- 172.21.15.7:0/2224933939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004102950 msgr2=0x7fd004197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd0037fe640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004102950 0x7fd004197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd0037fe640 1 -- 172.21.15.7:0/2224933939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff4009900 con 0x7fd004101760 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd003fff640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd004103310 0x7fd00419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.958+0000 7fd0037fe640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd004101760 0x7fd004197770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcff4000f80 tx=0x7fcff4034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.959+0000 7fd000ff9640 1 -- 172.21.15.7:0/2224933939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40329b0 con 0x7fd004101760 2024-03-23T13:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.959+0000 7fd000ff9640 1 -- 172.21.15.7:0/2224933939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff4003e50 con 0x7fd004101760 2024-03-23T13:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.959+0000 7fd009d77640 1 -- 172.21.15.7:0/2224933939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00419f810 con 0x7fd004101760 2024-03-23T13:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.959+0000 7fd000ff9640 1 -- 172.21.15.7:0/2224933939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4040450 con 0x7fd004101760 2024-03-23T13:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.959+0000 7fd009d77640 1 -- 172.21.15.7:0/2224933939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00419fc90 con 0x7fd004101760 2024-03-23T13:20:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd000ff9640 1 -- 172.21.15.7:0/2224933939 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fcff400a040 con 0x7fd004101760 2024-03-23T13:20:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd00ca2c740 1 Processor -- start 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd00ca2c740 1 -- start start 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 0x55827912ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 0x55827912fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558279130350 0x5582791327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5582790e61b0 con 0x558279130350 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558279098f80 con 0x55827912a7f0 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5582790964b0 con 0x55827912b1e0 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd000ff9640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcfec0a2160 0x7fcfec0a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd003fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 0x55827912ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd002ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558279130350 0x5582791327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd0037fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 0x55827912fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd003fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 0x55827912ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:56176/0 (socket says 172.21.15.7:56176) 2024-03-23T13:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd002ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558279130350 0x5582791327e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48104/0 (socket says 172.21.15.7:48104) 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd003fff640 1 -- 172.21.15.7:0/534614978 learned_addr learned my addr 172.21.15.7:0/534614978 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd003fff640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcfec0a2160 0x7fcfec0a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.962+0000 7fd000ff9640 1 -- 172.21.15.7:0/2224933939 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcff40f0a50 con 0x7fd004101760 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fd0037fe640 1 -- 172.21.15.7:0/534614978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 msgr2=0x55827912ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fd0037fe640 1 --2- 172.21.15.7:0/534614978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 0x55827912ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fd0037fe640 1 -- 172.21.15.7:0/534614978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558279130350 msgr2=0x5582791327e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fd0037fe640 1 --2- 172.21.15.7:0/534614978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558279130350 0x5582791327e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fd0037fe640 1 -- 172.21.15.7:0/534614978 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558279027810 con 0x55827912b1e0 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fd002ffd640 1 --2- 172.21.15.7:0/534614978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558279130350 0x5582791327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fd003fff640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcfec0a2160 0x7fcfec0a4620 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55827912ba40 tx=0x7fcffc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fd0037fe640 1 --2- 172.21.15.7:0/534614978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 0x55827912fce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcff40907c0 tx=0x7fcff40907f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=973b1695f3a5b007 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fcfe9ffb640 1 -- 172.21.15.7:0/534614978 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40dbea0 con 0x55827912b1e0 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fcfe9ffb640 1 -- 172.21.15.7:0/534614978 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff40a8110 con 0x55827912b1e0 2024-03-23T13:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.963+0000 7fcfe9ffb640 1 -- 172.21.15.7:0/534614978 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40c1880 con 0x55827912b1e0 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 -- 172.21.15.7:0/534614978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 msgr2=0x55827912fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/534614978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 0x55827912fce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcff40907c0 tx=0x7fcff40907f0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 -- 172.21.15.7:0/534614978 shutdown_connections 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/534614978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558279130350 0x5582791327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/534614978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 0x55827912fce0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/534614978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 0x55827912ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 -- 172.21.15.7:0/534614978 >> 172.21.15.7:0/534614978 conn(0x558279125d60 msgr2=0x558279126160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 -- 172.21.15.7:0/534614978 shutdown_connections 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 -- 172.21.15.7:0/534614978 wait complete. 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 -- start start 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.964+0000 7fd00ca2c740 1 Processor -- start 2024-03-23T13:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 0x558279134610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279134b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558279130350 0x55827913bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558279091c90 con 0x55827912b1e0 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558279036c50 con 0x55827912a7f0 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558278d65250 con 0x558279130350 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd00ca2c740 10 client.0 _ll_register_callbacks cb 0x558279085290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd002ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279134b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd002ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279134b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48110/0 (socket says 172.21.15.7:48110) 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd002ffd640 1 -- 172.21.15.7:0/524826629 learned_addr learned my addr 172.21.15.7:0/524826629 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd003fff640 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558279130350 0x55827913bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd0037fe640 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 0x558279134610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd002ffd640 1 -- 172.21.15.7:0/524826629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558279130350 msgr2=0x55827913bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd002ffd640 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558279130350 0x55827913bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd002ffd640 1 -- 172.21.15.7:0/524826629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 msgr2=0x558279134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd002ffd640 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 0x558279134610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd002ffd640 1 -- 172.21.15.7:0/524826629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00419fc90 con 0x55827912b1e0 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd003fff640 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558279130350 0x55827913bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd0037fe640 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 0x558279134610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fd002ffd640 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279134b50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcff8012720 tx=0x7fcff8012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d7fe612b9c2944 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.965+0000 7fcfe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/524826629 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8017070 con 0x55827912b1e0 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/524826629 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcff8004050 con 0x55827912b1e0 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fd00ca2c740 1 -- 172.21.15.7:0/524826629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558278d91600 con 0x55827912b1e0 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/524826629 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcff8022030 con 0x55827912b1e0 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fd00ca2c740 10 client.15438 fetch_fsmap learned FSMap version 29 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fd00ca2c740 1 -- 172.21.15.7:0/524826629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558279090310 con 0x55827912b1e0 2024-03-23T13:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/524826629 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558278d91600 con 0x55827912b1e0 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/524826629 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fcff80038c0 con 0x55827912b1e0 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fd00ca2c740 10 client.15438 fetch_fsmap finished waiting for FSMap version 29 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fd00ca2c740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fd00ca2c740 20 client.15438 populate_metadata read hostname 'smithi007' 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558279140650 0x558279142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fd00ca2c740 4 client.15438 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.966+0000 7fd00ca2c740 1 -- 172.21.15.7:0/524826629 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558278de21b0 con 0x558279140650 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.967+0000 7fd0037fe640 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558279140650 0x558279142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.967+0000 7fd0037fe640 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558279140650 0x558279142a70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1e6518ba8884dd server_cookie=69cbe6536bcf0367 in_seq=0 out_seq=0 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.967+0000 7fcfe2ffd640 10 client.15438 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/524826629 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558278de21b0 con 0x558279140650 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fcfe2ffd640 10 client.15438 handle_command_reply: tid=0 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 client.15438 shutdown 2024-03-23T13:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 -- 172.21.15.7:0/524826629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 msgr2=0x558279134b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279134b50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcff8012720 tx=0x7fcff8012b20 comp rx=0 tx=0).stop 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 -- 172.21.15.7:0/524826629 shutdown_connections 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x558279140650 0x558279142a70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558279130350 0x55827913bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279134b50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/524826629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55827912a7f0 0x558279134610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fcfe2ffd640 0 client.15438 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 -- 172.21.15.7:0/524826629 >> 172.21.15.7:0/524826629 conn(0x558279125d60 msgr2=0x558279124ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 -- 172.21.15.7:0/524826629 shutdown_connections 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 1 -- 172.21.15.7:0/524826629 wait complete. 2024-03-23T13:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.968+0000 7fd00ca2c740 20 client.15438 trim_cache size 0 max 16384 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd00ca2c740 1 Processor -- start 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd00ca2c740 1 -- start start 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912a7f0 0x558279126140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 0x55827913bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558279130350 0x558279134610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558279017660 con 0x55827912a7f0 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558278d91600 con 0x558279130350 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5582791330c0 con 0x55827912b1e0 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd0037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912a7f0 0x558279126140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd0037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912a7f0 0x558279126140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48112/0 (socket says 172.21.15.7:48112) 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd0037fe640 1 -- 172.21.15.7:0/2747080382 learned_addr learned my addr 172.21.15.7:0/2747080382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd002ffd640 1 --2- 172.21.15.7:0/2747080382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 0x55827913bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.983+0000 7fd003fff640 1 --2- 172.21.15.7:0/2747080382 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558279130350 0x558279134610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd0037fe640 1 -- 172.21.15.7:0/2747080382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 msgr2=0x55827913bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd0037fe640 1 --2- 172.21.15.7:0/2747080382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 0x55827913bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd0037fe640 1 -- 172.21.15.7:0/2747080382 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558279130350 msgr2=0x558279134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd0037fe640 1 --2- 172.21.15.7:0/2747080382 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558279130350 0x558279134610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd0037fe640 1 -- 172.21.15.7:0/2747080382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55827902c810 con 0x55827912a7f0 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd0037fe640 1 --2- 172.21.15.7:0/2747080382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912a7f0 0x558279126140 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcff40b9b90 tx=0x7fcff4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d1bb1e2d642c7e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fcfe97fa640 1 -- 172.21.15.7:0/2747080382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40dbea0 con 0x55827912a7f0 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fcfe97fa640 1 -- 172.21.15.7:0/2747080382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff40c5f50 con 0x55827912a7f0 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fcfe97fa640 1 -- 172.21.15.7:0/2747080382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40040d0 con 0x55827912a7f0 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd00ca2c740 1 -- 172.21.15.7:0/2747080382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912a7f0 msgr2=0x558279126140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/2747080382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912a7f0 0x558279126140 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcff40b9b90 tx=0x7fcff4009bf0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd00ca2c740 1 -- 172.21.15.7:0/2747080382 shutdown_connections 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/2747080382 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558279130350 0x558279134610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/2747080382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912b1e0 0x55827913bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/2747080382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912a7f0 0x558279126140 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.984+0000 7fd00ca2c740 1 -- 172.21.15.7:0/2747080382 >> 172.21.15.7:0/2747080382 conn(0x558279125d60 msgr2=0x558279120b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 1 -- 172.21.15.7:0/2747080382 shutdown_connections 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 1 -- 172.21.15.7:0/2747080382 wait complete. 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 1 -- start start 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 1 Processor -- start 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912a7f0 0x558279138c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279139180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558279130350 0x55827913a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55827913a660 con 0x55827912b1e0 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55827913a7d0 con 0x558279130350 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55827913a990 con 0x55827912a7f0 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd00ca2c740 10 client.0 _ll_register_callbacks cb 0x7fcff8009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd002ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd0037fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912a7f0 0x558279138c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd002ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279139180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48116/0 (socket says 172.21.15.7:48116) 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd002ffd640 1 -- 172.21.15.7:0/1723753178 learned_addr learned my addr 172.21.15.7:0/1723753178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.985+0000 7fd003fff640 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558279130350 0x55827913a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fd0037fe640 1 -- 172.21.15.7:0/1723753178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558279130350 msgr2=0x55827913a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fd0037fe640 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558279130350 0x55827913a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fd0037fe640 1 -- 172.21.15.7:0/1723753178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 msgr2=0x558279139180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fd0037fe640 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279139180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fd0037fe640 1 -- 172.21.15.7:0/1723753178 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558279027810 con 0x55827912a7f0 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fd002ffd640 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279139180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fd0037fe640 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912a7f0 0x558279138c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x558279142d40 tx=0x7fcff402c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bed82e37ca394be server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fcfe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/1723753178 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40dbea0 con 0x55827912a7f0 2024-03-23T13:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fd00ca2c740 1 -- 172.21.15.7:0/1723753178 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5582790b6790 con 0x55827912a7f0 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/1723753178 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcff4032b10 con 0x55827912a7f0 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/1723753178 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcff40c1880 con 0x55827912a7f0 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fd00ca2c740 10 client.25297 fetch_fsmap learned FSMap version 29 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.986+0000 7fd00ca2c740 1 -- 172.21.15.7:0/1723753178 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582790525a0 con 0x55827912a7f0 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/1723753178 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5582790b6790 con 0x55827912a7f0 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/1723753178 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fcff40bda20 con 0x55827912a7f0 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fd00ca2c740 10 client.25297 fetch_fsmap finished waiting for FSMap version 29 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fd00ca2c740 10 client.25297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fd00ca2c740 20 client.25297 populate_metadata read hostname 'smithi007' 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5582791a6970 0x5582791a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fd00ca2c740 4 client.25297 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fd00ca2c740 1 -- 172.21.15.7:0/1723753178 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558278dce550 con 0x5582791a6970 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fd002ffd640 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5582791a6970 0x5582791a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fcfe2ffd640 10 client.25297 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:46.987+0000 7fd002ffd640 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5582791a6970 0x5582791a8d60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9745c78a6f49f43f server_cookie=2402c406d31ca59d in_seq=0 out_seq=0 2024-03-23T13:20:47.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:47 smithi007 ceph-mon[32184]: pgmap v379: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 586 KiB/s wr, 140 op/s 2024-03-23T13:20:47.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:47 smithi130 ceph-mon[36225]: pgmap v379: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 586 KiB/s wr, 140 op/s 2024-03-23T13:20:47.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:47 smithi155 ceph-mon[36157]: pgmap v379: 97 pgs: 97 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 586 KiB/s wr, 140 op/s 2024-03-23T13:20:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.659+0000 7fcfe2ffd640 1 -- 172.21.15.7:0/1723753178 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558278dce550 con 0x5582791a6970 2024-03-23T13:20:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.659+0000 7fcfe2ffd640 10 client.25297 handle_command_reply: tid=0 2024-03-23T13:20:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.659+0000 7fd00ca2c740 1 client.25297 shutdown 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 1 -- 172.21.15.7:0/1723753178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912a7f0 msgr2=0x558279138c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912a7f0 0x558279138c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x558279142d40 tx=0x7fcff402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 1 -- 172.21.15.7:0/1723753178 shutdown_connections 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5582791a6970 0x5582791a8d60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558279130350 0x55827913a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55827912b1e0 0x558279139180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 1 --2- 172.21.15.7:0/1723753178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55827912a7f0 0x558279138c40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fcfe2ffd640 0 client.25297 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 1 -- 172.21.15.7:0/1723753178 >> 172.21.15.7:0/1723753178 conn(0x558279125d60 msgr2=0x558279126140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 1 -- 172.21.15.7:0/1723753178 shutdown_connections 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 1 -- 172.21.15.7:0/1723753178 wait complete. 2024-03-23T13:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd00ca2c740 20 client.25297 trim_cache size 0 max 16384 2024-03-23T13:20:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-23T13:20:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-23T13:20:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-23T13:20:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd009d77640 1 -- 172.21.15.7:0/2224933939 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcfec0a2160 msgr2=0x7fcfec0a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd009d77640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcfec0a2160 0x7fcfec0a4620 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55827912ba40 tx=0x7fcffc005f90 comp rx=0 tx=0).stop 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd009d77640 1 -- 172.21.15.7:0/2224933939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd004101760 msgr2=0x7fd004197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd009d77640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd004101760 0x7fd004197770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcff4000f80 tx=0x7fcff4034a90 comp rx=0 tx=0).stop 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd009d77640 1 -- 172.21.15.7:0/2224933939 shutdown_connections 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd009d77640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcfec0a2160 0x7fcfec0a4620 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd009d77640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd004103310 0x7fd00419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd009d77640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004102950 0x7fd004197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd009d77640 1 --2- 172.21.15.7:0/2224933939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd004101760 0x7fd004197770 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.660+0000 7fd009d77640 1 -- 172.21.15.7:0/2224933939 >> 172.21.15.7:0/2224933939 conn(0x7fd0040fccf0 msgr2=0x7fd004105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.661+0000 7fd009d77640 1 -- 172.21.15.7:0/2224933939 shutdown_connections 2024-03-23T13:20:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:20:47.661+0000 7fd009d77640 1 -- 172.21.15.7:0/2224933939 wait complete. 2024-03-23T13:20:49.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:49 smithi007 ceph-mon[32184]: pgmap v380: 97 pgs: 97 active+clean; 15 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 592 KiB/s wr, 194 op/s 2024-03-23T13:20:49.341 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:49 smithi155 ceph-mon[36157]: pgmap v380: 97 pgs: 97 active+clean; 15 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 592 KiB/s wr, 194 op/s 2024-03-23T13:20:49.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:49 smithi130 ceph-mon[36225]: pgmap v380: 97 pgs: 97 active+clean; 15 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 592 KiB/s wr, 194 op/s 2024-03-23T13:20:49.947 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 7% (321/4277) Updating files: 8% (343/4277) Updating files: 9% (385/4277) Updating files: 10% (428/4277) Updating files: 11% (471/4277) Updating files: 11% (489/4277) Updating files: 12% (514/4277) Updating files: 13% (557/4277) Updating files: 14% (599/4277) Updating files: 15% (642/4277) Updating files: 16% (685/4277) Updating files: 16% (689/4277) Updating files: 17% (728/4277) Updating files: 18% (770/4277) Updating files: 19% (813/4277) Updating files: 20% (856/4277) Updating files: 21% (899/4277) Updating files: 22% (941/4277) Updating files: 23% (984/4277) Updating files: 24% (1027/4277) Updating files: 25% (1070/4277) Updating files: 26% (1113/4277) Updating files: 27% (1155/4277) Updating files: 28% (1198/4277) Updating files: 29% (1241/4277) Updating files: 29% (1283/4277) Updating files: 30% (1284/4277) Updating files: 31% (1326/4277) Updating files: 32% (1369/4277) Updating files: 33% (1412/4277) Updating files: 34% (1455/4277) Updating files: 35% (1497/4277) Updating files: 36% (1540/4277) Updating files: 37% (1583/4277) Updating files: 38% (1626/4277) Updating files: 39% (1669/4277) Updating files: 40% (1711/4277) Updating files: 41% (1754/4277) Updating files: 42% (1797/4277) Updating files: 43% (1840/4277) Updating files: 43% (1869/4277) Updating files: 44% (1882/4277) Updating files: 45% (1925/4277) Updating files: 46% (1968/4277) Updating files: 47% (2011/4277) Updating files: 48% (2053/4277) Updating files: 49% (2096/4277) Updating files: 50% (2139/4277) Updating files: 51% (2182/4277) Updating files: 52% (2225/4277) Updating files: 52% (2263/4277) Updating files: 53% (2267/4277) Updating files: 54% (2310/4277) Updating files: 55% (2353/4277) Updating files: 56% (2396/4277) Updating files: 57% (2438/4277) Updating files: 58% (2481/4277) Updating files: 59% (2524/4277) Updating files: 60% (2567/4277) Updating files: 61% (2609/4277) Updating files: 62% (2652/4277) Updating files: 63% (2695/4277) Updating files: 64% (2738/4277) Updating files: 64% (2773/4277) Updating files: 65% (2781/4277) Updating files: 66% (2823/4277) Updating files: 67% (2866/4277) Updating files: 68% (2909/4277) Updating files: 69% (2952/4277) Updating files: 70% (2994/4277) Updating files: 70% (3011/4277) Updating files: 71% (3037/4277) Updating files: 72% (3080/4277) Updating files: 73% (3123/4277) Updating files: 74% (3165/4277) Updating files: 75% (3208/4277) Updating files: 76% (3251/4277) Updating files: 76% (3260/4277) Updating files: 77% (3294/4277) Updating files: 78% (3337/4277) Updating files: 79% (3379/4277) Updating files: 80% (3422/4277) Updating files: 81% (3465/4277) Updating files: 82% (3508/4277) Updating files: 83% (3550/4277) Updating files: 84% (3593/4277) Updating files: 85% (3636/4277) Updating files: 86% (3679/4277) Updating files: 87% (3721/4277) Updating files: 88% (3764/4277) Updating files: 89% (3807/4277) Updating files: 90% (3850/4277) Updating files: 91% (3893/4277) Updating files: 92% (3935/4277) Updating files: 93% (3978/4277) Updating files: 93% (3983/4277) Updating files: 94% (4021/4277) Updating files: 95% (4064/4277) Updating files: 96% (4106/4277) Updating files: 97% (4149/4277) Updating files: 98% (4192/4277) Updating files: 99% (4235/4277) Updating files: 100% (4277/4277) Updating files: 100% (4277/4277), done. 2024-03-23T13:20:49.955 INFO:tasks.workunit.client.0.smithi007.stderr:+ cd xfstests-dev 2024-03-23T13:20:49.955 INFO:tasks.workunit.client.0.smithi007.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-03-23T13:20:51.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:51 smithi007 ceph-mon[32184]: pgmap v381: 97 pgs: 97 active+clean; 15 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 389 KiB/s wr, 233 op/s 2024-03-23T13:20:51.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:51 smithi130 ceph-mon[36225]: pgmap v381: 97 pgs: 97 active+clean; 15 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 389 KiB/s wr, 233 op/s 2024-03-23T13:20:51.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:51 smithi155 ceph-mon[36157]: pgmap v381: 97 pgs: 97 active+clean; 15 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 389 KiB/s wr, 233 op/s 2024-03-23T13:20:53.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:53 smithi007 ceph-mon[32184]: pgmap v382: 97 pgs: 97 active+clean; 19 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 828 KiB/s wr, 495 op/s 2024-03-23T13:20:53.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:53 smithi007 ceph-mon[32184]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:20:53.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:53 smithi130 ceph-mon[36225]: pgmap v382: 97 pgs: 97 active+clean; 19 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 828 KiB/s wr, 495 op/s 2024-03-23T13:20:53.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:53 smithi130 ceph-mon[36225]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:20:53.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:53 smithi155 ceph-mon[36157]: pgmap v382: 97 pgs: 97 active+clean; 19 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 828 KiB/s wr, 495 op/s 2024-03-23T13:20:53.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:53 smithi155 ceph-mon[36157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T13:20:55.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:55 smithi007 ceph-mon[32184]: pgmap v383: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.2 MiB/s wr, 609 op/s 2024-03-23T13:20:55.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:55 smithi130 ceph-mon[36225]: pgmap v383: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.2 MiB/s wr, 609 op/s 2024-03-23T13:20:55.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:55 smithi155 ceph-mon[36157]: pgmap v383: 97 pgs: 97 active+clean; 25 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.2 MiB/s wr, 609 op/s 2024-03-23T13:20:57.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:57 smithi130 ceph-mon[36225]: pgmap v384: 97 pgs: 97 active+clean; 31 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.7 MiB/s wr, 816 op/s 2024-03-23T13:20:57.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:57 smithi155 ceph-mon[36157]: pgmap v384: 97 pgs: 97 active+clean; 31 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.7 MiB/s wr, 816 op/s 2024-03-23T13:20:57.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:57 smithi007 ceph-mon[32184]: pgmap v384: 97 pgs: 97 active+clean; 31 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.7 MiB/s wr, 816 op/s 2024-03-23T13:20:59.381 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:20:59 smithi007 ceph-mon[32184]: pgmap v385: 97 pgs: 97 active+clean; 41 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 881 op/s 2024-03-23T13:20:59.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:20:59 smithi130 ceph-mon[36225]: pgmap v385: 97 pgs: 97 active+clean; 41 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 881 op/s 2024-03-23T13:20:59.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:20:59 smithi155 ceph-mon[36157]: pgmap v385: 97 pgs: 97 active+clean; 41 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 881 op/s 2024-03-23T13:21:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.750+0000 7fcdb3d93640 1 -- 172.21.15.7:0/915406266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdac102270 msgr2=0x7fcdac104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.750+0000 7fcdb3d93640 1 --2- 172.21.15.7:0/915406266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdac102270 0x7fcdac104660 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009900 tx=0x7fcda802bc30 comp rx=0 tx=0).stop 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 -- 172.21.15.7:0/915406266 shutdown_connections 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 --2- 172.21.15.7:0/915406266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdac1074d0 0x7fcdac1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 --2- 172.21.15.7:0/915406266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcdac104ba0 0x7fcdac106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 --2- 172.21.15.7:0/915406266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdac102270 0x7fcdac104660 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 -- 172.21.15.7:0/915406266 >> 172.21.15.7:0/915406266 conn(0x7fcdac0fbba0 msgr2=0x7fcdac0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 -- 172.21.15.7:0/915406266 shutdown_connections 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 -- 172.21.15.7:0/915406266 wait complete. 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 Processor -- start 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 -- start start 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcdac102270 0x7fcdac101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdac104ba0 0x7fcdac100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdac1074d0 0x7fcdac100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.751+0000 7fcdb3d93640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac100ec0 con 0x7fcdac1074d0 2024-03-23T13:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb3d93640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac101030 con 0x7fcdac102270 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb3d93640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac1a17c0 con 0x7fcdac104ba0 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb2330640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdac1074d0 0x7fcdac100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb2330640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdac1074d0 0x7fcdac100760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60738/0 (socket says 172.21.15.7:60738) 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb2330640 1 -- 172.21.15.7:0/630214140 learned_addr learned my addr 172.21.15.7:0/630214140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb132e640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdac104ba0 0x7fcdac100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcdac102270 0x7fcdac101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb2330640 1 -- 172.21.15.7:0/630214140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdac104ba0 msgr2=0x7fcdac100220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb2330640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdac104ba0 0x7fcdac100220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb2330640 1 -- 172.21.15.7:0/630214140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcdac102270 msgr2=0x7fcdac101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb2330640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcdac102270 0x7fcdac101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb2330640 1 -- 172.21.15.7:0/630214140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda8009560 con 0x7fcdac1074d0 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb2330640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdac1074d0 0x7fcdac100760 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcda400eb40 tx=0x7fcda400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcda2ffd640 1 -- 172.21.15.7:0/630214140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4004b20 con 0x7fcdac1074d0 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb3d93640 1 -- 172.21.15.7:0/630214140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdac1a1a70 con 0x7fcdac1074d0 2024-03-23T13:21:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcda2ffd640 1 -- 172.21.15.7:0/630214140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda4004c80 con 0x7fcdac1074d0 2024-03-23T13:21:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcda2ffd640 1 -- 172.21.15.7:0/630214140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4010690 con 0x7fcdac1074d0 2024-03-23T13:21:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.752+0000 7fcdb3d93640 1 -- 172.21.15.7:0/630214140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdac1a1f80 con 0x7fcdac1074d0 2024-03-23T13:21:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.755+0000 7fcdb6974740 1 Processor -- start 2024-03-23T13:21:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.755+0000 7fcda2ffd640 1 -- 172.21.15.7:0/630214140 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fcda4005020 con 0x7fcdac1074d0 2024-03-23T13:21:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb6974740 1 -- start start 2024-03-23T13:21:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 0x55f6b8e26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 0x55f6b8e2bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 0x55f6b8e2e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb6974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8de21b0 con 0x55f6b8e2c350 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb6974740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8d94f80 con 0x55f6b8e271e0 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb6974740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8d924b0 con 0x55f6b8e267f0 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb132e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 0x55f6b8e26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb132e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 0x55f6b8e26c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:50878/0 (socket says 172.21.15.7:50878) 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb132e640 1 -- 172.21.15.7:0/1562037988 learned_addr learned my addr 172.21.15.7:0/1562037988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb1b2f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 0x55f6b8e2bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcdb132e640 1 --2- 172.21.15.7:0/1562037988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 0x55f6b8e2e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.756+0000 7fcda2ffd640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcd980a2160 0x7fcd980a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcdb132e640 1 -- 172.21.15.7:0/1562037988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 msgr2=0x55f6b8e2bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcdb132e640 1 --2- 172.21.15.7:0/1562037988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 0x55f6b8e2bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcdb132e640 1 -- 172.21.15.7:0/1562037988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 msgr2=0x55f6b8e2e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcdb132e640 1 --2- 172.21.15.7:0/1562037988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 0x55f6b8e2e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcdb132e640 1 -- 172.21.15.7:0/1562037988 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b8d23810 con 0x55f6b8e267f0 2024-03-23T13:21:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcdb2330640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcd980a2160 0x7fcd980a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/1562037988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 0x55f6b8e2bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:21:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcda2ffd640 1 -- 172.21.15.7:0/630214140 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcda4014070 con 0x7fcdac1074d0 2024-03-23T13:21:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcdb132e640 1 --2- 172.21.15.7:0/1562037988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 0x55f6b8e26c10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0058c0 tx=0x7fcd9c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f54ddc87c05f2512 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcd97fff640 1 -- 172.21.15.7:0/1562037988 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c004030 con 0x55f6b8e267f0 2024-03-23T13:21:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcd97fff640 1 -- 172.21.15.7:0/1562037988 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9c005020 con 0x55f6b8e267f0 2024-03-23T13:21:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcd97fff640 1 -- 172.21.15.7:0/1562037988 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c004c00 con 0x55f6b8e267f0 2024-03-23T13:21:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.757+0000 7fcdb2330640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcd980a2160 0x7fcd980a4620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcda40a3300 tx=0x7fcda406af40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.758+0000 7fcdb6974740 1 -- 172.21.15.7:0/1562037988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 msgr2=0x55f6b8e26c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.758+0000 7fcdb6974740 1 --2- 172.21.15.7:0/1562037988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 0x55f6b8e26c10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0058c0 tx=0x7fcd9c005850 comp rx=0 tx=0).stop 2024-03-23T13:21:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.758+0000 7fcdb6974740 1 -- 172.21.15.7:0/1562037988 shutdown_connections 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.758+0000 7fcdb6974740 1 --2- 172.21.15.7:0/1562037988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 0x55f6b8e2e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.758+0000 7fcdb6974740 1 --2- 172.21.15.7:0/1562037988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 0x55f6b8e2bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.758+0000 7fcdb6974740 1 --2- 172.21.15.7:0/1562037988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 0x55f6b8e26c10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.758+0000 7fcdb6974740 1 -- 172.21.15.7:0/1562037988 >> 172.21.15.7:0/1562037988 conn(0x55f6b8e21d60 msgr2=0x55f6b8e22160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.758+0000 7fcdb6974740 1 -- 172.21.15.7:0/1562037988 shutdown_connections 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.758+0000 7fcdb6974740 1 -- 172.21.15.7:0/1562037988 wait complete. 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.758+0000 7fcdb6974740 1 -- start start 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb6974740 1 Processor -- start 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e267f0 0x55f6b8e30800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e271e0 0x55f6b8e30d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e2c350 0x55f6b8e37d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb6974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8d8dc90 con 0x55f6b8e267f0 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb6974740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8d32c50 con 0x55f6b8e2c350 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb6974740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8a61250 con 0x55f6b8e271e0 2024-03-23T13:21:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb6974740 10 client.0 _ll_register_callbacks cb 0x55f6b8d81290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb132e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e267f0 0x55f6b8e30800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb132e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e267f0 0x55f6b8e30800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60768/0 (socket says 172.21.15.7:60768) 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb132e640 1 -- 172.21.15.7:0/4062959395 learned_addr learned my addr 172.21.15.7:0/4062959395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e271e0 0x55f6b8e30d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb132e640 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e2c350 0x55f6b8e37d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb132e640 1 -- 172.21.15.7:0/4062959395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e271e0 msgr2=0x55f6b8e30d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb132e640 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e271e0 0x55f6b8e30d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb132e640 1 -- 172.21.15.7:0/4062959395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e2c350 msgr2=0x55f6b8e37d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb132e640 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e2c350 0x55f6b8e37d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.759+0000 7fcdb132e640 1 -- 172.21.15.7:0/4062959395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c019040 con 0x55f6b8e267f0 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e271e0 0x55f6b8e30d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcdb132e640 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e267f0 0x55f6b8e30800 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00a010 tx=0x7fcd9c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d19018495246d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcd957fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcd957fa640 1 -- 172.21.15.7:0/4062959395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c004940 con 0x55f6b8e267f0 2024-03-23T13:21:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcd957fa640 1 -- 172.21.15.7:0/4062959395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcd9c004aa0 con 0x55f6b8e267f0 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcdb6974740 1 -- 172.21.15.7:0/4062959395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f6b8db2790 con 0x55f6b8e267f0 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcd957fa640 1 -- 172.21.15.7:0/4062959395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd9c004c60 con 0x55f6b8e267f0 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcdb6974740 10 client.15456 fetch_fsmap learned FSMap version 30 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcdb6974740 1 -- 172.21.15.7:0/4062959395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f6b8d8c310 con 0x55f6b8e267f0 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcd957fa640 1 -- 172.21.15.7:0/4062959395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f6b8db2790 con 0x55f6b8e267f0 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcd957fa640 1 -- 172.21.15.7:0/4062959395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcd9c01d070 con 0x55f6b8e267f0 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcdb6974740 10 client.15456 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcdb6974740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.760+0000 7fcdb6974740 20 client.15456 populate_metadata read hostname 'smithi007' 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.761+0000 7fcdb6974740 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55f6b8e3c670 0x55f6b8e3ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.761+0000 7fcdb6974740 4 client.15456 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.761+0000 7fcdb6974740 1 -- 172.21.15.7:0/4062959395 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6b8a8d600 con 0x55f6b8e3c670 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.761+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55f6b8e3c670 0x55f6b8e3ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.761+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55f6b8e3c670 0x55f6b8e3ea90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18aa8d1a6781b41d server_cookie=ae9f73f1126e96f9 in_seq=0 out_seq=0 2024-03-23T13:21:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.761+0000 7fcd957fa640 10 client.15456 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.762+0000 7fcd957fa640 1 -- 172.21.15.7:0/4062959395 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f6b8a8d600 con 0x55f6b8e3c670 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.762+0000 7fcd957fa640 10 client.15456 handle_command_reply: tid=0 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.762+0000 7fcdb6974740 1 client.15456 shutdown 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 1 -- 172.21.15.7:0/4062959395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e267f0 msgr2=0x55f6b8e30800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e267f0 0x55f6b8e30800 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00a010 tx=0x7fcd9c009da0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 1 -- 172.21.15.7:0/4062959395 shutdown_connections 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55f6b8e3c670 0x55f6b8e3ea90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e2c350 0x55f6b8e37d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e271e0 0x55f6b8e30d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcd957fa640 0 client.15456 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 1 --2- 172.21.15.7:0/4062959395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e267f0 0x55f6b8e30800 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 1 -- 172.21.15.7:0/4062959395 >> 172.21.15.7:0/4062959395 conn(0x55f6b8e21d60 msgr2=0x55f6b8e20ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 1 -- 172.21.15.7:0/4062959395 shutdown_connections 2024-03-23T13:21:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 1 -- 172.21.15.7:0/4062959395 wait complete. 2024-03-23T13:21:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.763+0000 7fcdb6974740 20 client.15456 trim_cache size 0 max 16384 2024-03-23T13:21:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb6974740 1 Processor -- start 2024-03-23T13:21:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb6974740 1 -- start start 2024-03-23T13:21:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e267f0 0x55f6b8e30ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e271e0 0x55f6b8e37d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e2c350 0x55f6b8e20610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb6974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8d13660 con 0x55f6b8e271e0 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb6974740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8db27c0 con 0x55f6b8e267f0 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb6974740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8e38ee0 con 0x55f6b8e2c350 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb1b2f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e271e0 0x55f6b8e37d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb1b2f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e271e0 0x55f6b8e37d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60778/0 (socket says 172.21.15.7:60778) 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb1b2f640 1 -- 172.21.15.7:0/3129949485 learned_addr learned my addr 172.21.15.7:0/3129949485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb132e640 1 --2- 172.21.15.7:0/3129949485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e2c350 0x55f6b8e20610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb132e640 1 --2- 172.21.15.7:0/3129949485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e267f0 0x55f6b8e30ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb1b2f640 1 -- 172.21.15.7:0/3129949485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e2c350 msgr2=0x55f6b8e20610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/3129949485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e2c350 0x55f6b8e20610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb1b2f640 1 -- 172.21.15.7:0/3129949485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e267f0 msgr2=0x55f6b8e30ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/3129949485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e267f0 0x55f6b8e30ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.778+0000 7fcdb1b2f640 1 -- 172.21.15.7:0/3129949485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b8d28810 con 0x55f6b8e271e0 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.779+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/3129949485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e271e0 0x55f6b8e37d40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcda802c170 tx=0x7fcda8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96fdc16f7f8c2c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.779+0000 7fcd977fe640 1 -- 172.21.15.7:0/3129949485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8004840 con 0x55f6b8e271e0 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.779+0000 7fcd977fe640 1 -- 172.21.15.7:0/3129949485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda80049a0 con 0x55f6b8e271e0 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.779+0000 7fcd977fe640 1 -- 172.21.15.7:0/3129949485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda804a620 con 0x55f6b8e271e0 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 -- 172.21.15.7:0/3129949485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e271e0 msgr2=0x55f6b8e37d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 --2- 172.21.15.7:0/3129949485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e271e0 0x55f6b8e37d40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcda802c170 tx=0x7fcda8005fd0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 -- 172.21.15.7:0/3129949485 shutdown_connections 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 --2- 172.21.15.7:0/3129949485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e2c350 0x55f6b8e20610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 --2- 172.21.15.7:0/3129949485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e271e0 0x55f6b8e37d40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 --2- 172.21.15.7:0/3129949485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e267f0 0x55f6b8e30ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 -- 172.21.15.7:0/3129949485 >> 172.21.15.7:0/3129949485 conn(0x55f6b8e21d60 msgr2=0x55f6b8e22140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 -- 172.21.15.7:0/3129949485 shutdown_connections 2024-03-23T13:21:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 -- 172.21.15.7:0/3129949485 wait complete. 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 -- start start 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 Processor -- start 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 0x55f6b8e35ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 0x55f6b8e361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 0x55f6b8e371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8e37730 con 0x55f6b8e2c350 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8e378a0 con 0x55f6b8e271e0 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8e37a60 con 0x55f6b8e267f0 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb6974740 10 client.0 _ll_register_callbacks cb 0x7fcd9c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb132e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 0x55f6b8e371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.780+0000 7fcdb1b2f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 0x55f6b8e361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb132e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 0x55f6b8e371f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60786/0 (socket says 172.21.15.7:60786) 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb1b2f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 0x55f6b8e361e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:36948/0 (socket says 172.21.15.7:36948) 2024-03-23T13:21:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb132e640 1 -- 172.21.15.7:0/3342327824 learned_addr learned my addr 172.21.15.7:0/3342327824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb132e640 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 0x55f6b8e35ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb132e640 1 -- 172.21.15.7:0/3342327824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 msgr2=0x55f6b8e35ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb132e640 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 0x55f6b8e35ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb132e640 1 -- 172.21.15.7:0/3342327824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 msgr2=0x55f6b8e361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb132e640 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 0x55f6b8e361e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb132e640 1 -- 172.21.15.7:0/3342327824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda8030070 con 0x55f6b8e2c350 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb132e640 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 0x55f6b8e371f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00fd80 tx=0x7fcd9c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7325c25d9b93027 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcd957fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcd957fa640 1 -- 172.21.15.7:0/3342327824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c009a90 con 0x55f6b8e2c350 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcd957fa640 1 -- 172.21.15.7:0/3342327824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcd9c0097a0 con 0x55f6b8e2c350 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb6974740 1 -- 172.21.15.7:0/3342327824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd9c005020 con 0x55f6b8e2c350 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcd957fa640 1 -- 172.21.15.7:0/3342327824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd9c00fea0 con 0x55f6b8e2c350 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb6974740 10 client.15462 fetch_fsmap learned FSMap version 30 2024-03-23T13:21:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcdb6974740 1 -- 172.21.15.7:0/3342327824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f6b8d4e5a0 con 0x55f6b8e2c350 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.781+0000 7fcd957fa640 1 -- 172.21.15.7:0/3342327824 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9c005020 con 0x55f6b8e2c350 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.782+0000 7fcd957fa640 1 -- 172.21.15.7:0/3342327824 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcd9c01d040 con 0x55f6b8e2c350 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.782+0000 7fcdb6974740 10 client.15462 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.782+0000 7fcdb6974740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.782+0000 7fcdb6974740 20 client.15462 populate_metadata read hostname 'smithi007' 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.782+0000 7fcdb6974740 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55f6b8ea3960 0x55f6b8ea5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.782+0000 7fcdb6974740 4 client.15462 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.782+0000 7fcdb6974740 1 -- 172.21.15.7:0/3342327824 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6b8ade1b0 con 0x55f6b8ea3960 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.782+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55f6b8ea3960 0x55f6b8ea5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.782+0000 7fcdb1b2f640 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55f6b8ea3960 0x55f6b8ea5d80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2184f9843c1f51d6 server_cookie=af2711669f353bfb in_seq=0 out_seq=0 2024-03-23T13:21:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:00.782+0000 7fcd957fa640 10 client.15462 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:21:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcd957fa640 1 -- 172.21.15.7:0/3342327824 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6b8ade1b0 con 0x55f6b8ea3960 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcd957fa640 10 client.15462 handle_command_reply: tid=0 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 client.15462 shutdown 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 -- 172.21.15.7:0/3342327824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 msgr2=0x55f6b8e371f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 0x55f6b8e371f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00fd80 tx=0x7fcd9c0052f0 comp rx=0 tx=0).stop 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 -- 172.21.15.7:0/3342327824 shutdown_connections 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x55f6b8ea3960 0x55f6b8ea5d80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6b8e2c350 0x55f6b8e371f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f6b8e271e0 0x55f6b8e361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcd957fa640 0 client.15462 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 --2- 172.21.15.7:0/3342327824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6b8e267f0 0x55f6b8e35ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 -- 172.21.15.7:0/3342327824 >> 172.21.15.7:0/3342327824 conn(0x55f6b8e21d60 msgr2=0x55f6b8e20610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 -- 172.21.15.7:0/3342327824 shutdown_connections 2024-03-23T13:21:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 1 -- 172.21.15.7:0/3342327824 wait complete. 2024-03-23T13:21:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb6974740 20 client.15462 trim_cache size 0 max 16384 2024-03-23T13:21:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-23T13:21:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-23T13:21:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-23T13:21:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-23T13:21:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb3d93640 1 -- 172.21.15.7:0/630214140 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcd980a2160 msgr2=0x7fcd980a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb3d93640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcd980a2160 0x7fcd980a4620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcda40a3300 tx=0x7fcda406af40 comp rx=0 tx=0).stop 2024-03-23T13:21:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb3d93640 1 -- 172.21.15.7:0/630214140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdac1074d0 msgr2=0x7fcdac100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb3d93640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdac1074d0 0x7fcdac100760 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcda400eb40 tx=0x7fcda400c2d0 comp rx=0 tx=0).stop 2024-03-23T13:21:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb3d93640 1 -- 172.21.15.7:0/630214140 shutdown_connections 2024-03-23T13:21:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb3d93640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fcd980a2160 0x7fcd980a4620 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb3d93640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdac1074d0 0x7fcdac100760 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb3d93640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdac104ba0 0x7fcdac100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.022+0000 7fcdb3d93640 1 --2- 172.21.15.7:0/630214140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcdac102270 0x7fcdac101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.023+0000 7fcdb3d93640 1 -- 172.21.15.7:0/630214140 >> 172.21.15.7:0/630214140 conn(0x7fcdac0fbba0 msgr2=0x7fcdac102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.023+0000 7fcdb3d93640 1 -- 172.21.15.7:0/630214140 shutdown_connections 2024-03-23T13:21:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:01.023+0000 7fcdb3d93640 1 -- 172.21.15.7:0/630214140 wait complete. 2024-03-23T13:21:01.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:01 smithi130 ceph-mon[36225]: pgmap v386: 97 pgs: 97 active+clean; 41 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 790 op/s 2024-03-23T13:21:01.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:01 smithi155 ceph-mon[36157]: pgmap v386: 97 pgs: 97 active+clean; 41 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 790 op/s 2024-03-23T13:21:01.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:01 smithi007 ceph-mon[32184]: pgmap v386: 97 pgs: 97 active+clean; 41 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 790 op/s 2024-03-23T13:21:03.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:03 smithi130 ceph-mon[36225]: pgmap v387: 97 pgs: 97 active+clean; 43 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 779 op/s 2024-03-23T13:21:03.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:03 smithi155 ceph-mon[36157]: pgmap v387: 97 pgs: 97 active+clean; 43 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 779 op/s 2024-03-23T13:21:03.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:03 smithi007 ceph-mon[32184]: pgmap v387: 97 pgs: 97 active+clean; 43 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 779 op/s 2024-03-23T13:21:04.330 INFO:tasks.workunit.client.0.smithi007.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-03-23T13:21:04.330 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:21:04.330 INFO:tasks.workunit.client.0.smithi007.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr:state without impacting any branches by switching back to a branch. 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr: git switch -c 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr:Or undo this operation with: 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr: git switch - 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-23T13:21:04.331 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-23T13:21:04.332 INFO:tasks.workunit.client.0.smithi007.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-03-23T13:21:04.334 INFO:tasks.workunit.client.0.smithi007.stderr:+ make -j4 2024-03-23T13:21:04.341 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize -cfi 2024-03-23T13:21:04.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:04 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:21:04.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:04 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:21:04.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:04 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:21:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:04 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:21:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:04 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:21:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:04 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:21:04.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:04 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:21:04.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:04 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:21:04.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:04 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:21:04.745 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: putting auxiliary files in '.'. 2024-03-23T13:21:04.745 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: copying file './config.guess' 2024-03-23T13:21:04.750 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: copying file './config.sub' 2024-03-23T13:21:04.755 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: copying file './install-sh' 2024-03-23T13:21:04.760 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: copying file './ltmain.sh' 2024-03-23T13:21:04.784 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-03-23T13:21:04.784 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-03-23T13:21:04.790 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-03-23T13:21:04.796 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-03-23T13:21:04.802 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-03-23T13:21:04.807 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-03-23T13:21:04.813 INFO:tasks.workunit.client.0.smithi007.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-03-23T13:21:04.815 INFO:tasks.workunit.client.0.smithi007.stdout:cp include/install-sh . 2024-03-23T13:21:05.005 INFO:tasks.workunit.client.0.smithi007.stdout:aclocal -I m4 2024-03-23T13:21:05.347 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:05 smithi130 ceph-mon[36225]: pgmap v388: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 597 op/s 2024-03-23T13:21:05.347 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:05 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:21:05.347 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:05 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:21:05.347 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:05 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:21:05.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:05 smithi155 ceph-mon[36157]: pgmap v388: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 597 op/s 2024-03-23T13:21:05.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:05 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:21:05.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:05 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:21:05.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:05 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:21:05.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:05 smithi007 ceph-mon[32184]: pgmap v388: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 597 op/s 2024-03-23T13:21:05.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:21:05.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:21:05.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:21:06.116 INFO:tasks.workunit.client.0.smithi007.stdout:autoheader 2024-03-23T13:21:06.430 INFO:tasks.workunit.client.0.smithi007.stdout:autoconf 2024-03-23T13:21:06.612 INFO:tasks.workunit.client.0.smithi007.stdout:./configure \ 2024-03-23T13:21:06.612 INFO:tasks.workunit.client.0.smithi007.stdout: --libexecdir=/usr/lib \ 2024-03-23T13:21:06.612 INFO:tasks.workunit.client.0.smithi007.stdout: --exec_prefix=/var/lib 2024-03-23T13:21:06.737 INFO:tasks.workunit.client.0.smithi007.stdout:checking build system type... x86_64-pc-linux-gnu 2024-03-23T13:21:06.737 INFO:tasks.workunit.client.0.smithi007.stdout:checking host system type... x86_64-pc-linux-gnu 2024-03-23T13:21:06.738 INFO:tasks.workunit.client.0.smithi007.stdout:checking how to print strings... printf 2024-03-23T13:21:06.738 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc... gcc 2024-03-23T13:21:06.827 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether the C compiler works... yes 2024-03-23T13:21:06.827 INFO:tasks.workunit.client.0.smithi007.stdout:checking for C compiler default output file name... a.out 2024-03-23T13:21:06.861 INFO:tasks.workunit.client.0.smithi007.stdout:checking for suffix of executables... 2024-03-23T13:21:06.898 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether we are cross compiling... no 2024-03-23T13:21:06.935 INFO:tasks.workunit.client.0.smithi007.stdout:checking for suffix of object files... o 2024-03-23T13:21:06.962 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether we are using the GNU C compiler... yes 2024-03-23T13:21:06.986 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether gcc accepts -g... yes 2024-03-23T13:21:07.075 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc option to accept ISO C89... none needed 2024-03-23T13:21:07.079 INFO:tasks.workunit.client.0.smithi007.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-03-23T13:21:07.084 INFO:tasks.workunit.client.0.smithi007.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-03-23T13:21:07.085 INFO:tasks.workunit.client.0.smithi007.stdout:checking for egrep... /usr/bin/grep -E 2024-03-23T13:21:07.087 INFO:tasks.workunit.client.0.smithi007.stdout:checking for fgrep... /usr/bin/grep -F 2024-03-23T13:21:07.090 INFO:tasks.workunit.client.0.smithi007.stdout:checking for ld used by gcc... /usr/bin/ld 2024-03-23T13:21:07.091 INFO:tasks.workunit.client.0.smithi007.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-03-23T13:21:07.096 INFO:tasks.workunit.client.0.smithi007.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-03-23T13:21:07.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:07 smithi130 ceph-mon[36225]: pgmap v389: 97 pgs: 97 active+clean; 30 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 593 op/s 2024-03-23T13:21:07.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:07 smithi155 ceph-mon[36157]: pgmap v389: 97 pgs: 97 active+clean; 30 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 593 op/s 2024-03-23T13:21:07.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:07 smithi007 ceph-mon[32184]: pgmap v389: 97 pgs: 97 active+clean; 30 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 593 op/s 2024-03-23T13:21:08.070 INFO:tasks.workunit.client.0.smithi007.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-03-23T13:21:08.070 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether ln -s works... yes 2024-03-23T13:21:08.077 INFO:tasks.workunit.client.0.smithi007.stdout:checking the maximum length of command line arguments... 1572864 2024-03-23T13:21:08.080 INFO:tasks.workunit.client.0.smithi007.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-03-23T13:21:08.080 INFO:tasks.workunit.client.0.smithi007.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-03-23T13:21:08.080 INFO:tasks.workunit.client.0.smithi007.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-03-23T13:21:08.080 INFO:tasks.workunit.client.0.smithi007.stdout:checking for objdump... objdump 2024-03-23T13:21:08.080 INFO:tasks.workunit.client.0.smithi007.stdout:checking how to recognize dependent libraries... pass_all 2024-03-23T13:21:08.081 INFO:tasks.workunit.client.0.smithi007.stdout:checking for dlltool... no 2024-03-23T13:21:08.081 INFO:tasks.workunit.client.0.smithi007.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-03-23T13:21:08.081 INFO:tasks.workunit.client.0.smithi007.stdout:checking for ar... ar 2024-03-23T13:21:08.179 INFO:tasks.workunit.client.0.smithi007.stdout:checking for archiver @FILE support... @ 2024-03-23T13:21:08.180 INFO:tasks.workunit.client.0.smithi007.stdout:checking for strip... strip 2024-03-23T13:21:08.180 INFO:tasks.workunit.client.0.smithi007.stdout:checking for ranlib... ranlib 2024-03-23T13:21:08.180 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gawk... gawk 2024-03-23T13:21:08.285 INFO:tasks.workunit.client.0.smithi007.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-03-23T13:21:08.287 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sysroot... no 2024-03-23T13:21:08.302 INFO:tasks.workunit.client.0.smithi007.stdout:checking for a working dd... /usr/bin/dd 2024-03-23T13:21:08.316 INFO:tasks.workunit.client.0.smithi007.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-03-23T13:21:08.351 INFO:tasks.workunit.client.0.smithi007.stdout:checking for mt... no 2024-03-23T13:21:08.363 INFO:tasks.workunit.client.0.smithi007.stdout:checking if : is a manifest tool... no 2024-03-23T13:21:08.459 INFO:tasks.workunit.client.0.smithi007.stdout:checking how to run the C preprocessor... gcc -E 2024-03-23T13:21:08.673 INFO:tasks.workunit.client.0.smithi007.stdout:checking for ANSI C header files... yes 2024-03-23T13:21:08.710 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/types.h... yes 2024-03-23T13:21:08.755 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/stat.h... yes 2024-03-23T13:21:08.797 INFO:tasks.workunit.client.0.smithi007.stdout:checking for stdlib.h... yes 2024-03-23T13:21:08.841 INFO:tasks.workunit.client.0.smithi007.stdout:checking for string.h... yes 2024-03-23T13:21:08.889 INFO:tasks.workunit.client.0.smithi007.stdout:checking for memory.h... yes 2024-03-23T13:21:08.934 INFO:tasks.workunit.client.0.smithi007.stdout:checking for strings.h... yes 2024-03-23T13:21:08.978 INFO:tasks.workunit.client.0.smithi007.stdout:checking for inttypes.h... yes 2024-03-23T13:21:09.024 INFO:tasks.workunit.client.0.smithi007.stdout:checking for stdint.h... yes 2024-03-23T13:21:09.069 INFO:tasks.workunit.client.0.smithi007.stdout:checking for unistd.h... yes 2024-03-23T13:21:09.172 INFO:tasks.workunit.client.0.smithi007.stdout:checking for dlfcn.h... yes 2024-03-23T13:21:09.178 INFO:tasks.workunit.client.0.smithi007.stdout:checking for objdir... .libs 2024-03-23T13:21:09.311 INFO:tasks.workunit.client.0.smithi007.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-03-23T13:21:09.311 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-03-23T13:21:09.350 INFO:tasks.workunit.client.0.smithi007.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-03-23T13:21:09.397 INFO:tasks.workunit.client.0.smithi007.stdout:checking if gcc static flag -static works... no 2024-03-23T13:21:09.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:09 smithi130 ceph-mon[36225]: pgmap v390: 97 pgs: 97 active+clean; 31 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 417 op/s 2024-03-23T13:21:09.456 INFO:tasks.workunit.client.0.smithi007.stdout:checking if gcc supports -c -o file.o... yes 2024-03-23T13:21:09.456 INFO:tasks.workunit.client.0.smithi007.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-03-23T13:21:09.462 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-03-23T13:21:09.532 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether -lc should be explicitly linked in... no 2024-03-23T13:21:09.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:09 smithi155 ceph-mon[36157]: pgmap v390: 97 pgs: 97 active+clean; 31 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 417 op/s 2024-03-23T13:21:09.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:09 smithi007 ceph-mon[32184]: pgmap v390: 97 pgs: 97 active+clean; 31 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 417 op/s 2024-03-23T13:21:09.606 INFO:tasks.workunit.client.0.smithi007.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-03-23T13:21:09.606 INFO:tasks.workunit.client.0.smithi007.stdout:checking how to hardcode library paths into programs... immediate 2024-03-23T13:21:09.607 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether stripping libraries is possible... yes 2024-03-23T13:21:09.607 INFO:tasks.workunit.client.0.smithi007.stdout:checking if libtool supports shared libraries... yes 2024-03-23T13:21:09.607 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether to build shared libraries... yes 2024-03-23T13:21:09.607 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether to build static libraries... yes 2024-03-23T13:21:09.614 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc... (cached) gcc 2024-03-23T13:21:09.649 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-03-23T13:21:09.649 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether gcc accepts -g... (cached) yes 2024-03-23T13:21:09.649 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-03-23T13:21:09.649 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gmake... /usr/bin/gmake 2024-03-23T13:21:09.649 INFO:tasks.workunit.client.0.smithi007.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-03-23T13:21:09.650 INFO:tasks.workunit.client.0.smithi007.stdout:checking for tar... /bin/tar 2024-03-23T13:21:09.650 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gzip... /bin/gzip 2024-03-23T13:21:09.670 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether gcc -MM is supported... yes 2024-03-23T13:21:09.671 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sort... /bin/sort 2024-03-23T13:21:09.671 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether ln -s works... yes 2024-03-23T13:21:09.671 INFO:tasks.workunit.client.0.smithi007.stdout:checking for rpm... /bin/rpm 2024-03-23T13:21:09.674 INFO:tasks.workunit.client.0.smithi007.stdout:checking for rpmbuild... no 2024-03-23T13:21:09.674 INFO:tasks.workunit.client.0.smithi007.stdout:checking for ANSI C header files... (cached) yes 2024-03-23T13:21:09.740 INFO:tasks.workunit.client.0.smithi007.stdout:checking assert.h usability... yes 2024-03-23T13:21:09.761 INFO:tasks.workunit.client.0.smithi007.stdout:checking assert.h presence... yes 2024-03-23T13:21:09.761 INFO:tasks.workunit.client.0.smithi007.stdout:checking for assert.h... yes 2024-03-23T13:21:09.823 INFO:tasks.workunit.client.0.smithi007.stdout:checking bstring.h usability... no 2024-03-23T13:21:09.855 INFO:tasks.workunit.client.0.smithi007.stdout:checking bstring.h presence... no 2024-03-23T13:21:09.855 INFO:tasks.workunit.client.0.smithi007.stdout:checking for bstring.h... no 2024-03-23T13:21:09.892 INFO:tasks.workunit.client.0.smithi007.stdout:checking libgen.h usability... yes 2024-03-23T13:21:09.908 INFO:tasks.workunit.client.0.smithi007.stdout:checking libgen.h presence... yes 2024-03-23T13:21:09.909 INFO:tasks.workunit.client.0.smithi007.stdout:checking for libgen.h... yes 2024-03-23T13:21:09.972 INFO:tasks.workunit.client.0.smithi007.stdout:checking dirent.h usability... yes 2024-03-23T13:21:09.992 INFO:tasks.workunit.client.0.smithi007.stdout:checking dirent.h presence... yes 2024-03-23T13:21:09.992 INFO:tasks.workunit.client.0.smithi007.stdout:checking for dirent.h... yes 2024-03-23T13:21:10.043 INFO:tasks.workunit.client.0.smithi007.stdout:checking errno.h usability... yes 2024-03-23T13:21:10.064 INFO:tasks.workunit.client.0.smithi007.stdout:checking errno.h presence... yes 2024-03-23T13:21:10.064 INFO:tasks.workunit.client.0.smithi007.stdout:checking for errno.h... yes 2024-03-23T13:21:10.123 INFO:tasks.workunit.client.0.smithi007.stdout:checking malloc.h usability... yes 2024-03-23T13:21:10.146 INFO:tasks.workunit.client.0.smithi007.stdout:checking malloc.h presence... yes 2024-03-23T13:21:10.146 INFO:tasks.workunit.client.0.smithi007.stdout:checking for malloc.h... yes 2024-03-23T13:21:10.209 INFO:tasks.workunit.client.0.smithi007.stdout:checking uuid.h usability... no 2024-03-23T13:21:10.247 INFO:tasks.workunit.client.0.smithi007.stdout:checking uuid.h presence... no 2024-03-23T13:21:10.247 INFO:tasks.workunit.client.0.smithi007.stdout:checking for uuid.h... no 2024-03-23T13:21:10.310 INFO:tasks.workunit.client.0.smithi007.stdout:checking uuid/uuid.h usability... yes 2024-03-23T13:21:10.332 INFO:tasks.workunit.client.0.smithi007.stdout:checking uuid/uuid.h presence... yes 2024-03-23T13:21:10.332 INFO:tasks.workunit.client.0.smithi007.stdout:checking for uuid/uuid.h... yes 2024-03-23T13:21:10.388 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/uuid.h usability... no 2024-03-23T13:21:10.422 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/uuid.h presence... no 2024-03-23T13:21:10.422 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/uuid.h... no 2024-03-23T13:21:10.466 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/file.h usability... yes 2024-03-23T13:21:10.488 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/file.h presence... yes 2024-03-23T13:21:10.488 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/file.h... yes 2024-03-23T13:21:10.543 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/fcntl.h usability... yes 2024-03-23T13:21:10.560 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/fcntl.h presence... yes 2024-03-23T13:21:10.561 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/fcntl.h... yes 2024-03-23T13:21:10.617 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/syssgi.h usability... no 2024-03-23T13:21:10.648 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/syssgi.h presence... no 2024-03-23T13:21:10.648 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/syssgi.h... no 2024-03-23T13:21:10.746 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/param.h usability... yes 2024-03-23T13:21:10.766 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/param.h presence... yes 2024-03-23T13:21:10.766 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/param.h... yes 2024-03-23T13:21:10.780 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/stat.h... (cached) yes 2024-03-23T13:21:10.826 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/statvfs.h usability... yes 2024-03-23T13:21:10.846 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/statvfs.h presence... yes 2024-03-23T13:21:10.846 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/statvfs.h... yes 2024-03-23T13:21:10.891 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/time.h usability... yes 2024-03-23T13:21:10.910 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/time.h presence... yes 2024-03-23T13:21:10.910 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/time.h... yes 2024-03-23T13:21:10.955 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/ioctl.h usability... yes 2024-03-23T13:21:10.971 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/ioctl.h presence... yes 2024-03-23T13:21:10.971 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/ioctl.h... yes 2024-03-23T13:21:11.033 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/wait.h usability... yes 2024-03-23T13:21:11.055 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/wait.h presence... yes 2024-03-23T13:21:11.055 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/wait.h... yes 2024-03-23T13:21:11.067 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/types.h... (cached) yes 2024-03-23T13:21:11.071 INFO:tasks.workunit.client.0.smithi007.stdout:checking for strings.h... (cached) yes 2024-03-23T13:21:11.106 INFO:tasks.workunit.client.0.smithi007.stdout:checking err.h usability... yes 2024-03-23T13:21:11.126 INFO:tasks.workunit.client.0.smithi007.stdout:checking err.h presence... yes 2024-03-23T13:21:11.126 INFO:tasks.workunit.client.0.smithi007.stdout:checking for err.h... yes 2024-03-23T13:21:11.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:11 smithi130 ceph-mon[36225]: pgmap v391: 97 pgs: 97 active+clean; 30 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 893 KiB/s wr, 363 op/s 2024-03-23T13:21:11.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:11 smithi155 ceph-mon[36157]: pgmap v391: 97 pgs: 97 active+clean; 30 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 893 KiB/s wr, 363 op/s 2024-03-23T13:21:11.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:11 smithi007 ceph-mon[32184]: pgmap v391: 97 pgs: 97 active+clean; 30 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 893 KiB/s wr, 363 op/s 2024-03-23T13:21:11.857 INFO:tasks.workunit.client.0.smithi007.stdout:checking linux/falloc.h usability... yes 2024-03-23T13:21:11.877 INFO:tasks.workunit.client.0.smithi007.stdout:checking linux/falloc.h presence... yes 2024-03-23T13:21:11.878 INFO:tasks.workunit.client.0.smithi007.stdout:checking for linux/falloc.h... yes 2024-03-23T13:21:12.174 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-03-23T13:21:12.237 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-03-23T13:21:12.238 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/fs/xfs_fsops.h... no 2024-03-23T13:21:12.312 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/fs/xfs_itable.h usability... no 2024-03-23T13:21:12.408 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/fs/xfs_itable.h presence... no 2024-03-23T13:21:12.408 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/fs/xfs_itable.h... no 2024-03-23T13:21:12.590 INFO:tasks.workunit.client.0.smithi007.stdout:checking xfs/platform_defs.h usability... no 2024-03-23T13:21:12.625 INFO:tasks.workunit.client.0.smithi007.stdout:checking xfs/platform_defs.h presence... no 2024-03-23T13:21:12.625 INFO:tasks.workunit.client.0.smithi007.stdout:checking for xfs/platform_defs.h... no 2024-03-23T13:21:12.672 INFO:tasks.workunit.client.0.smithi007.stdout:checking btrfs/ioctl.h usability... no 2024-03-23T13:21:12.706 INFO:tasks.workunit.client.0.smithi007.stdout:checking btrfs/ioctl.h presence... no 2024-03-23T13:21:12.706 INFO:tasks.workunit.client.0.smithi007.stdout:checking for btrfs/ioctl.h... no 2024-03-23T13:21:12.749 INFO:tasks.workunit.client.0.smithi007.stdout:checking cifs/ioctl.h usability... no 2024-03-23T13:21:12.784 INFO:tasks.workunit.client.0.smithi007.stdout:checking cifs/ioctl.h presence... no 2024-03-23T13:21:12.784 INFO:tasks.workunit.client.0.smithi007.stdout:checking for cifs/ioctl.h... no 2024-03-23T13:21:12.826 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/mman.h usability... yes 2024-03-23T13:21:12.843 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/mman.h presence... yes 2024-03-23T13:21:12.843 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/mman.h... yes 2024-03-23T13:21:12.883 INFO:tasks.workunit.client.0.smithi007.stdout:checking for xfs/xfs_log_format.h... no 2024-03-23T13:21:12.941 INFO:tasks.workunit.client.0.smithi007.stdout:checking for uuid_compare... no 2024-03-23T13:21:12.979 INFO:tasks.workunit.client.0.smithi007.stdout:checking for uuid_compare in -luuid... yes 2024-03-23T13:21:13.049 INFO:tasks.workunit.client.0.smithi007.stdout:checking for xfs/xfs.h... yes 2024-03-23T13:21:13.083 INFO:tasks.workunit.client.0.smithi007.stdout:checking for xfs/libxfs.h... no 2024-03-23T13:21:13.120 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether xlog_assign_lsn is declared... no 2024-03-23T13:21:13.158 INFO:tasks.workunit.client.0.smithi007.stdout:checking for xfs/xqm.h... yes 2024-03-23T13:21:13.210 INFO:tasks.workunit.client.0.smithi007.stdout:checking xfsctl from xfs/xfs.h... ok 2024-03-23T13:21:13.250 INFO:tasks.workunit.client.0.smithi007.stdout:checking xfs/handle.h usability... yes 2024-03-23T13:21:13.265 INFO:tasks.workunit.client.0.smithi007.stdout:checking xfs/handle.h presence... yes 2024-03-23T13:21:13.266 INFO:tasks.workunit.client.0.smithi007.stdout:checking for xfs/handle.h... yes 2024-03-23T13:21:13.305 INFO:tasks.workunit.client.0.smithi007.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-03-23T13:21:13.374 INFO:tasks.workunit.client.0.smithi007.stdout:checking attr/attributes.h usability... yes 2024-03-23T13:21:13.392 INFO:tasks.workunit.client.0.smithi007.stdout:checking attr/attributes.h presence... yes 2024-03-23T13:21:13.392 INFO:tasks.workunit.client.0.smithi007.stdout:checking for attr/attributes.h... yes 2024-03-23T13:21:13.444 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/attributes.h usability... no 2024-03-23T13:21:13.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:13 smithi130 ceph-mon[36225]: pgmap v392: 97 pgs: 97 active+clean; 33 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 411 op/s 2024-03-23T13:21:13.486 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/attributes.h presence... no 2024-03-23T13:21:13.486 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/attributes.h... no 2024-03-23T13:21:13.527 INFO:tasks.workunit.client.0.smithi007.stdout:checking for attr_list in -lattr... yes 2024-03-23T13:21:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:13 smithi155 ceph-mon[36157]: pgmap v392: 97 pgs: 97 active+clean; 33 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 411 op/s 2024-03-23T13:21:13.568 INFO:tasks.workunit.client.0.smithi007.stdout:checking for attr_set in -lattr... yes 2024-03-23T13:21:13.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:13 smithi007 ceph-mon[32184]: pgmap v392: 97 pgs: 97 active+clean; 33 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 411 op/s 2024-03-23T13:21:13.600 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/acl.h usability... yes 2024-03-23T13:21:13.618 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/acl.h presence... yes 2024-03-23T13:21:13.619 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/acl.h... yes 2024-03-23T13:21:13.649 INFO:tasks.workunit.client.0.smithi007.stdout:checking acl/libacl.h usability... yes 2024-03-23T13:21:13.674 INFO:tasks.workunit.client.0.smithi007.stdout:checking acl/libacl.h presence... yes 2024-03-23T13:21:13.674 INFO:tasks.workunit.client.0.smithi007.stdout:checking for acl/libacl.h... yes 2024-03-23T13:21:13.718 INFO:tasks.workunit.client.0.smithi007.stdout:checking for acl_init in -lacl... yes 2024-03-23T13:21:13.802 INFO:tasks.workunit.client.0.smithi007.stdout:checking gdbm-ndbm.h usability... no 2024-03-23T13:21:13.832 INFO:tasks.workunit.client.0.smithi007.stdout:checking gdbm-ndbm.h presence... no 2024-03-23T13:21:13.832 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gdbm-ndbm.h... no 2024-03-23T13:21:13.874 INFO:tasks.workunit.client.0.smithi007.stdout:checking gdbm/ndbm.h usability... no 2024-03-23T13:21:13.908 INFO:tasks.workunit.client.0.smithi007.stdout:checking gdbm/ndbm.h presence... no 2024-03-23T13:21:13.908 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gdbm/ndbm.h... no 2024-03-23T13:21:13.953 INFO:tasks.workunit.client.0.smithi007.stdout:checking ndbm.h usability... no 2024-03-23T13:21:13.985 INFO:tasks.workunit.client.0.smithi007.stdout:checking ndbm.h presence... no 2024-03-23T13:21:13.986 INFO:tasks.workunit.client.0.smithi007.stdout:checking for ndbm.h... no 2024-03-23T13:21:14.021 INFO:tasks.workunit.client.0.smithi007.stdout:checking libaio.h usability... yes 2024-03-23T13:21:14.045 INFO:tasks.workunit.client.0.smithi007.stdout:checking libaio.h presence... yes 2024-03-23T13:21:14.045 INFO:tasks.workunit.client.0.smithi007.stdout:checking for libaio.h... yes 2024-03-23T13:21:14.096 INFO:tasks.workunit.client.0.smithi007.stdout:checking for dm_make_handle in -ldm... no 2024-03-23T13:21:14.096 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-03-23T13:21:14.096 INFO:tasks.workunit.client.0.smithi007.stdout:WARNING: could not find a valid DMAPI base library. 2024-03-23T13:21:14.096 INFO:tasks.workunit.client.0.smithi007.stdout:If you want DMAPI support please install the data migration 2024-03-23T13:21:14.097 INFO:tasks.workunit.client.0.smithi007.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-03-23T13:21:14.097 INFO:tasks.workunit.client.0.smithi007.stdout:from the dmapi source. 2024-03-23T13:21:14.097 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-03-23T13:21:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.114+0000 7fd59a0d4640 1 -- 172.21.15.7:0/3276248165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd594101820 msgr2=0x7fd594101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.114+0000 7fd59a0d4640 1 --2- 172.21.15.7:0/3276248165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd594101820 0x7fd594101ca0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009900 tx=0x7fd57c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:21:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.114+0000 7fd59a0d4640 1 -- 172.21.15.7:0/3276248165 shutdown_connections 2024-03-23T13:21:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.114+0000 7fd59a0d4640 1 --2- 172.21.15.7:0/3276248165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5941021e0 0x7fd594108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.114+0000 7fd59a0d4640 1 --2- 172.21.15.7:0/3276248165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd594101820 0x7fd594101ca0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.114+0000 7fd59a0d4640 1 --2- 172.21.15.7:0/3276248165 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd594100630 0x7fd594100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.114+0000 7fd59a0d4640 1 -- 172.21.15.7:0/3276248165 >> 172.21.15.7:0/3276248165 conn(0x7fd5940fbba0 msgr2=0x7fd5940fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.114+0000 7fd59a0d4640 1 -- 172.21.15.7:0/3276248165 shutdown_connections 2024-03-23T13:21:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.114+0000 7fd59a0d4640 1 -- 172.21.15.7:0/3276248165 wait complete. 2024-03-23T13:21:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd59a0d4640 1 Processor -- start 2024-03-23T13:21:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd59a0d4640 1 -- start start 2024-03-23T13:21:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd59a0d4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd594100630 0x7fd594197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd59a0d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd594101820 0x7fd594197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd59a0d4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5941021e0 0x7fd59419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd59a0d4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd59419f210 con 0x7fd594101820 2024-03-23T13:21:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd59a0d4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd59419f380 con 0x7fd594100630 2024-03-23T13:21:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd59a0d4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd59419f540 con 0x7fd5941021e0 2024-03-23T13:21:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd594101820 0x7fd594197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd594101820 0x7fd594197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44722/0 (socket says 172.21.15.7:44722) 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd592ffd640 1 -- 172.21.15.7:0/2086562875 learned_addr learned my addr 172.21.15.7:0/2086562875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd593fff640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5941021e0 0x7fd59419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd5937fe640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd594100630 0x7fd594197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd592ffd640 1 -- 172.21.15.7:0/2086562875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5941021e0 msgr2=0x7fd59419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd592ffd640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5941021e0 0x7fd59419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd592ffd640 1 -- 172.21.15.7:0/2086562875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd594100630 msgr2=0x7fd594197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd592ffd640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd594100630 0x7fd594197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd592ffd640 1 -- 172.21.15.7:0/2086562875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd584009840 con 0x7fd594101820 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.115+0000 7fd5937fe640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd594100630 0x7fd594197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.116+0000 7fd592ffd640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd594101820 0x7fd594197c60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd57c006010 tx=0x7fd57c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.116+0000 7fd590ff9640 1 -- 172.21.15.7:0/2086562875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0043d0 con 0x7fd594101820 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.116+0000 7fd59a0d4640 1 -- 172.21.15.7:0/2086562875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c009560 con 0x7fd594101820 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.116+0000 7fd590ff9640 1 -- 172.21.15.7:0/2086562875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57c0052d0 con 0x7fd594101820 2024-03-23T13:21:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.116+0000 7fd590ff9640 1 -- 172.21.15.7:0/2086562875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c003cc0 con 0x7fd594101820 2024-03-23T13:21:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.116+0000 7fd59a0d4640 1 -- 172.21.15.7:0/2086562875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59419fb90 con 0x7fd594101820 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.118+0000 7fd590ff9640 1 -- 172.21.15.7:0/2086562875 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fd57c005440 con 0x7fd594101820 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.118+0000 7fd59cd91740 1 Processor -- start 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd59cd91740 1 -- start start 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a5c00 0x5622408a6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd590ff9640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5800a2160 0x7fd5800a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a65f0 0x5622408ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408ab760 0x5622408adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd59cd91740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56224085d1b0 con 0x5622408a65f0 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd59cd91740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56224080ff80 con 0x5622408a5c00 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd59cd91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56224080d4b0 con 0x5622408ab760 2024-03-23T13:21:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a65f0 0x5622408ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a65f0 0x5622408ab0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44724/0 (socket says 172.21.15.7:44724) 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd5937fe640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5800a2160 0x7fd5800a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd592ffd640 1 -- 172.21.15.7:0/2630977729 learned_addr learned my addr 172.21.15.7:0/2630977729 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd593fff640 1 --2- 172.21.15.7:0/2630977729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a5c00 0x5622408a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd590ff9640 1 -- 172.21.15.7:0/2086562875 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd57c0eff10 con 0x7fd594101820 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd593fff640 1 --2- 172.21.15.7:0/2630977729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408ab760 0x5622408adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd593fff640 1 -- 172.21.15.7:0/2630977729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408ab760 msgr2=0x5622408adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd593fff640 1 --2- 172.21.15.7:0/2630977729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408ab760 0x5622408adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd593fff640 1 -- 172.21.15.7:0/2630977729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a65f0 msgr2=0x5622408ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd593fff640 1 --2- 172.21.15.7:0/2630977729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a65f0 0x5622408ab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd593fff640 1 -- 172.21.15.7:0/2630977729 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56224079e810 con 0x5622408a5c00 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd592ffd640 1 --2- 172.21.15.7:0/2630977729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a65f0 0x5622408ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd5937fe640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5800a2160 0x7fd5800a4620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd584009fd0 tx=0x7fd584009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.119+0000 7fd593fff640 1 --2- 172.21.15.7:0/2630977729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a5c00 0x5622408a6020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0116e0 tx=0x7fd58c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71b316c1829fa5a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd589ffb640 1 -- 172.21.15.7:0/2630977729 <== mon.2 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c004050 con 0x5622408a5c00 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd589ffb640 1 -- 172.21.15.7:0/2630977729 <== mon.2 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c00fca0 con 0x5622408a5c00 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd589ffb640 1 -- 172.21.15.7:0/2630977729 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c0038e0 con 0x5622408a5c00 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd59cd91740 1 -- 172.21.15.7:0/2630977729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a5c00 msgr2=0x5622408a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd59cd91740 1 --2- 172.21.15.7:0/2630977729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a5c00 0x5622408a6020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0116e0 tx=0x7fd58c004730 comp rx=0 tx=0).stop 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd59cd91740 1 -- 172.21.15.7:0/2630977729 shutdown_connections 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd59cd91740 1 --2- 172.21.15.7:0/2630977729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408ab760 0x5622408adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd59cd91740 1 --2- 172.21.15.7:0/2630977729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a65f0 0x5622408ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd59cd91740 1 --2- 172.21.15.7:0/2630977729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a5c00 0x5622408a6020 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd59cd91740 1 -- 172.21.15.7:0/2630977729 >> 172.21.15.7:0/2630977729 conn(0x56224089cd60 msgr2=0x56224089d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.120+0000 7fd59cd91740 1 -- 172.21.15.7:0/2630977729 shutdown_connections 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd59cd91740 1 -- 172.21.15.7:0/2630977729 wait complete. 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd59cd91740 1 -- start start 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd59cd91740 1 Processor -- start 2024-03-23T13:21:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a5c00 0x5622408afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a65f0 0x5622408b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408ab760 0x5622408b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd59cd91740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562240808c90 con 0x5622408ab760 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd59cd91740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5622407adc50 con 0x5622408a65f0 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd59cd91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5622404dc250 con 0x5622408a5c00 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd59cd91740 10 client.0 _ll_register_callbacks cb 0x5622407fc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd593fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408ab760 0x5622408b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd5937fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a5c00 0x5622408afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd593fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408ab760 0x5622408b7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44726/0 (socket says 172.21.15.7:44726) 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd593fff640 1 -- 172.21.15.7:0/2786023465 learned_addr learned my addr 172.21.15.7:0/2786023465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.121+0000 7fd592ffd640 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a65f0 0x5622408b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd593fff640 1 -- 172.21.15.7:0/2786023465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a5c00 msgr2=0x5622408afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd593fff640 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a5c00 0x5622408afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd593fff640 1 -- 172.21.15.7:0/2786023465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a65f0 msgr2=0x5622408b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd593fff640 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a65f0 0x5622408b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd593fff640 1 -- 172.21.15.7:0/2786023465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58c019040 con 0x5622408ab760 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd593fff640 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408ab760 0x5622408b7190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009be0 tx=0x7fd58c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69185206d302bd6c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd572ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd572ffd640 1 -- 172.21.15.7:0/2786023465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c011030 con 0x5622408ab760 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd572ffd640 1 -- 172.21.15.7:0/2786023465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd58c011190 con 0x5622408ab760 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd59cd91740 1 -- 172.21.15.7:0/2786023465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56224082d790 con 0x5622408ab760 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd572ffd640 1 -- 172.21.15.7:0/2786023465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd58c029430 con 0x5622408ab760 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd59cd91740 10 client.15474 fetch_fsmap learned FSMap version 30 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd59cd91740 1 -- 172.21.15.7:0/2786023465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562240807310 con 0x5622408ab760 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.122+0000 7fd572ffd640 1 -- 172.21.15.7:0/2786023465 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56224082d790 con 0x5622408ab760 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.123+0000 7fd572ffd640 1 -- 172.21.15.7:0/2786023465 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd58c032070 con 0x5622408ab760 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.123+0000 7fd59cd91740 10 client.15474 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T13:21:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.123+0000 7fd59cd91740 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:21:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.123+0000 7fd59cd91740 20 client.15474 populate_metadata read hostname 'smithi007' 2024-03-23T13:21:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.123+0000 7fd59cd91740 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5622408bb9f0 0x5622408bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.123+0000 7fd59cd91740 4 client.15474 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T13:21:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.123+0000 7fd59cd91740 1 -- 172.21.15.7:0/2786023465 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562240508600 con 0x5622408bb9f0 2024-03-23T13:21:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.123+0000 7fd5937fe640 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5622408bb9f0 0x5622408bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.123+0000 7fd5937fe640 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5622408bb9f0 0x5622408bde10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a7f59e5340ec83 server_cookie=1f4b74b6a889c3a7 in_seq=0 out_seq=0 2024-03-23T13:21:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.123+0000 7fd572ffd640 10 client.15474 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:21:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.125+0000 7fd572ffd640 1 -- 172.21.15.7:0/2786023465 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562240508600 con 0x5622408bb9f0 2024-03-23T13:21:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd572ffd640 10 client.15474 handle_command_reply: tid=0 2024-03-23T13:21:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 client.15474 shutdown 2024-03-23T13:21:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 -- 172.21.15.7:0/2786023465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408ab760 msgr2=0x5622408b7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408ab760 0x5622408b7190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009be0 tx=0x7fd58c005850 comp rx=0 tx=0).stop 2024-03-23T13:21:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 -- 172.21.15.7:0/2786023465 shutdown_connections 2024-03-23T13:21:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x5622408bb9f0 0x5622408bde10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408ab760 0x5622408b7190 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a65f0 0x5622408b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd572ffd640 0 client.15474 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:21:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 --2- 172.21.15.7:0/2786023465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a5c00 0x5622408afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 -- 172.21.15.7:0/2786023465 >> 172.21.15.7:0/2786023465 conn(0x56224089cd60 msgr2=0x56224089bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 -- 172.21.15.7:0/2786023465 shutdown_connections 2024-03-23T13:21:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 1 -- 172.21.15.7:0/2786023465 wait complete. 2024-03-23T13:21:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.126+0000 7fd59cd91740 20 client.15474 trim_cache size 0 max 16384 2024-03-23T13:21:14.133 INFO:tasks.workunit.client.0.smithi007.stdout:checking linux/fiemap.h usability... yes 2024-03-23T13:21:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.141+0000 7fd59cd91740 1 Processor -- start 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.141+0000 7fd59cd91740 1 -- start start 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.141+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x562240897d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a65f0 0x5622408b7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408ab760 0x5622408aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd59cd91740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5622408ab620 con 0x5622408a5c00 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd59cd91740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56224078e660 con 0x5622408ab760 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd59cd91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5622408b0e00 con 0x5622408a65f0 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a65f0 0x5622408b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x562240897d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x562240897d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44736/0 (socket says 172.21.15.7:44736) 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a65f0 0x5622408b7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:56414/0 (socket says 172.21.15.7:56414) 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5937fe640 1 -- 172.21.15.7:0/817366521 learned_addr learned my addr 172.21.15.7:0/817366521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd593fff640 1 --2- 172.21.15.7:0/817366521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408ab760 0x5622408aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5937fe640 1 -- 172.21.15.7:0/817366521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a65f0 msgr2=0x5622408b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5937fe640 1 --2- 172.21.15.7:0/817366521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a65f0 0x5622408b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5937fe640 1 -- 172.21.15.7:0/817366521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408ab760 msgr2=0x5622408aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5937fe640 1 --2- 172.21.15.7:0/817366521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408ab760 0x5622408aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5937fe640 1 -- 172.21.15.7:0/817366521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622407a3810 con 0x5622408a5c00 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd592ffd640 1 --2- 172.21.15.7:0/817366521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a65f0 0x5622408b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd593fff640 1 --2- 172.21.15.7:0/817366521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408ab760 0x5622408aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5937fe640 1 --2- 172.21.15.7:0/817366521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x562240897d40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd594101680 tx=0x7fd584014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2af9eb704ff50f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5897fa640 1 -- 172.21.15.7:0/817366521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584024070 con 0x5622408a5c00 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5897fa640 1 -- 172.21.15.7:0/817366521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd584003e50 con 0x5622408a5c00 2024-03-23T13:21:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.142+0000 7fd5897fa640 1 -- 172.21.15.7:0/817366521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58401fb40 con 0x5622408a5c00 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.143+0000 7fd59cd91740 1 -- 172.21.15.7:0/817366521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 msgr2=0x562240897d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.143+0000 7fd59cd91740 1 --2- 172.21.15.7:0/817366521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x562240897d40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd594101680 tx=0x7fd584014000 comp rx=0 tx=0).stop 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.143+0000 7fd59cd91740 1 -- 172.21.15.7:0/817366521 shutdown_connections 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.143+0000 7fd59cd91740 1 --2- 172.21.15.7:0/817366521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408ab760 0x5622408aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.143+0000 7fd59cd91740 1 --2- 172.21.15.7:0/817366521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408a65f0 0x5622408b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.143+0000 7fd59cd91740 1 --2- 172.21.15.7:0/817366521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x562240897d40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.143+0000 7fd59cd91740 1 -- 172.21.15.7:0/817366521 >> 172.21.15.7:0/817366521 conn(0x56224089cd60 msgr2=0x56224089d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.143+0000 7fd59cd91740 1 -- 172.21.15.7:0/817366521 shutdown_connections 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.143+0000 7fd59cd91740 1 -- 172.21.15.7:0/817366521 wait complete. 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.143+0000 7fd59cd91740 1 -- start start 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd59cd91740 1 Processor -- start 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x5622408b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a65f0 0x5622408b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd59cd91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408ab760 0x56224091fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd59cd91740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5622408b4aa0 con 0x5622408a5c00 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd59cd91740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5622408b4be0 con 0x5622408a65f0 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd59cd91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562240920260 con 0x5622408ab760 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd59cd91740 10 client.0 _ll_register_callbacks cb 0x7fd57c004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd5937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x5622408b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd5937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x5622408b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44744/0 (socket says 172.21.15.7:44744) 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd5937fe640 1 -- 172.21.15.7:0/3594053935 learned_addr learned my addr 172.21.15.7:0/3594053935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd593fff640 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408ab760 0x56224091fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd592ffd640 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a65f0 0x5622408b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd5937fe640 1 -- 172.21.15.7:0/3594053935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408ab760 msgr2=0x56224091fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd5937fe640 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408ab760 0x56224091fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd5937fe640 1 -- 172.21.15.7:0/3594053935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a65f0 msgr2=0x5622408b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T13:21:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd5937fe640 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a65f0 0x5622408b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd5937fe640 1 -- 172.21.15.7:0/3594053935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c009560 con 0x5622408a5c00 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd5937fe640 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x5622408b4020 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd584027310 tx=0x7fd584027340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f065ef6b26febea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.144+0000 7fd572ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd572ffd640 1 -- 172.21.15.7:0/3594053935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584024040 con 0x5622408a5c00 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd572ffd640 1 -- 172.21.15.7:0/3594053935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd584003b70 con 0x5622408a5c00 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd59cd91740 1 -- 172.21.15.7:0/3594053935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58c00fca0 con 0x5622408a5c00 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd572ffd640 1 -- 172.21.15.7:0/3594053935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd584037030 con 0x5622408a5c00 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd59cd91740 10 client.15480 fetch_fsmap learned FSMap version 30 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd59cd91740 1 -- 172.21.15.7:0/3594053935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562240807310 con 0x5622408a5c00 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd572ffd640 1 -- 172.21.15.7:0/3594053935 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c00fca0 con 0x5622408a5c00 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd572ffd640 1 -- 172.21.15.7:0/3594053935 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd58402f020 con 0x5622408a5c00 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd59cd91740 10 client.15480 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd59cd91740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd59cd91740 20 client.15480 populate_metadata read hostname 'smithi007' 2024-03-23T13:21:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd59cd91740 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x562240921da0 0x5622409241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd59cd91740 4 client.15480 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T13:21:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd59cd91740 1 -- 172.21.15.7:0/3594053935 --> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622405591b0 con 0x562240921da0 2024-03-23T13:21:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.145+0000 7fd592ffd640 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x562240921da0 0x5622409241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.146+0000 7fd592ffd640 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x562240921da0 0x5622409241c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1c975f17bb1493 server_cookie=90333933e516847d in_seq=0 out_seq=0 2024-03-23T13:21:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.146+0000 7fd572ffd640 10 client.15480 ms_handle_connect on v2:172.21.15.130:6838/791500055 2024-03-23T13:21:14.153 INFO:tasks.workunit.client.0.smithi007.stdout:checking linux/fiemap.h presence... yes 2024-03-23T13:21:14.154 INFO:tasks.workunit.client.0.smithi007.stdout:checking for linux/fiemap.h... yes 2024-03-23T13:21:14.189 INFO:tasks.workunit.client.0.smithi007.stdout:checking for fallocate... yes 2024-03-23T13:21:14.237 INFO:tasks.workunit.client.0.smithi007.stdout:checking for open_by_handle_at... yes 2024-03-23T13:21:14.293 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/prctl.h usability... yes 2024-03-23T13:21:14.310 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/prctl.h presence... yes 2024-03-23T13:21:14.310 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/prctl.h... yes 2024-03-23T13:21:14.342 INFO:tasks.workunit.client.0.smithi007.stdout:checking linux/fs.h usability... yes 2024-03-23T13:21:14.358 INFO:tasks.workunit.client.0.smithi007.stdout:checking linux/fs.h presence... yes 2024-03-23T13:21:14.358 INFO:tasks.workunit.client.0.smithi007.stdout:checking for linux/fs.h... yes 2024-03-23T13:21:14.423 INFO:tasks.workunit.client.0.smithi007.stdout:checking btrfsutil.h usability... no 2024-03-23T13:21:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.470+0000 7fd572ffd640 1 -- 172.21.15.7:0/3594053935 <== mds.0 v2:172.21.15.130:6838/791500055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622405591b0 con 0x562240921da0 2024-03-23T13:21:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.470+0000 7fd572ffd640 10 client.15480 handle_command_reply: tid=0 2024-03-23T13:21:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.470+0000 7fd59cd91740 1 client.15480 shutdown 2024-03-23T13:21:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.470+0000 7fd59cd91740 1 -- 172.21.15.7:0/3594053935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 msgr2=0x5622408b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.470+0000 7fd59cd91740 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x5622408b4020 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd584027310 tx=0x7fd584027340 comp rx=0 tx=0).stop 2024-03-23T13:21:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd59cd91740 1 -- 172.21.15.7:0/3594053935 shutdown_connections 2024-03-23T13:21:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd59cd91740 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.130:6838/791500055,v1:172.21.15.130:6839/791500055] conn(0x562240921da0 0x5622409241c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd59cd91740 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622408ab760 0x56224091fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd59cd91740 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5622408a65f0 0x5622408b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd59cd91740 1 --2- 172.21.15.7:0/3594053935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5622408a5c00 0x5622408b4020 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd59cd91740 1 -- 172.21.15.7:0/3594053935 >> 172.21.15.7:0/3594053935 conn(0x56224089cd60 msgr2=0x5622408aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:14.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd572ffd640 0 client.15480 ms_handle_reset on v2:172.21.15.130:6838/791500055 2024-03-23T13:21:14.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd59cd91740 1 -- 172.21.15.7:0/3594053935 shutdown_connections 2024-03-23T13:21:14.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd59cd91740 1 -- 172.21.15.7:0/3594053935 wait complete. 2024-03-23T13:21:14.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd59cd91740 20 client.15480 trim_cache size 0 max 16384 2024-03-23T13:21:14.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-23T13:21:14.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-23T13:21:14.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-23T13:21:14.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-23T13:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.471+0000 7fd59a0d4640 1 -- 172.21.15.7:0/2086562875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5800a2160 msgr2=0x7fd5800a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5800a2160 0x7fd5800a4620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd584009fd0 tx=0x7fd584009500 comp rx=0 tx=0).stop 2024-03-23T13:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 -- 172.21.15.7:0/2086562875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd594101820 msgr2=0x7fd594197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd594101820 0x7fd594197c60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd57c006010 tx=0x7fd57c005e80 comp rx=0 tx=0).stop 2024-03-23T13:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 -- 172.21.15.7:0/2086562875 shutdown_connections 2024-03-23T13:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fd5800a2160 0x7fd5800a4620 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5941021e0 0x7fd59419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd594101820 0x7fd594197c60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 --2- 172.21.15.7:0/2086562875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd594100630 0x7fd594197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 -- 172.21.15.7:0/2086562875 >> 172.21.15.7:0/2086562875 conn(0x7fd5940fbba0 msgr2=0x7fd5940fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 -- 172.21.15.7:0/2086562875 shutdown_connections 2024-03-23T13:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-23T13:21:14.472+0000 7fd59a0d4640 1 -- 172.21.15.7:0/2086562875 wait complete. 2024-03-23T13:21:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:checking btrfsutil.h presence... no 2024-03-23T13:21:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:checking for btrfsutil.h... no 2024-03-23T13:21:14.539 INFO:tasks.workunit.client.0.smithi007.stdout:checking for copy_file_range... yes 2024-03-23T13:21:14.588 INFO:tasks.workunit.client.0.smithi007.stdout:checking for renameat2... yes 2024-03-23T13:21:14.608 INFO:tasks.workunit.client.0.smithi007.stdout:configure: creating ./config.status 2024-03-23T13:21:14.946 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: creating include/builddefs 2024-03-23T13:21:15.446 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:15 smithi130 ceph-mon[36225]: pgmap v393: 97 pgs: 97 active+clean; 37 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.3 MiB/s wr, 387 op/s 2024-03-23T13:21:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:15 smithi155 ceph-mon[36157]: pgmap v393: 97 pgs: 97 active+clean; 37 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.3 MiB/s wr, 387 op/s 2024-03-23T13:21:15.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:15 smithi007 ceph-mon[32184]: pgmap v393: 97 pgs: 97 active+clean; 37 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.3 MiB/s wr, 387 op/s 2024-03-23T13:21:17.039 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: creating include/config.h 2024-03-23T13:21:17.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:17 smithi130 ceph-mon[36225]: pgmap v394: 97 pgs: 97 active+clean; 39 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.8 MiB/s wr, 426 op/s 2024-03-23T13:21:17.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:17 smithi155 ceph-mon[36157]: pgmap v394: 97 pgs: 97 active+clean; 39 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.8 MiB/s wr, 426 op/s 2024-03-23T13:21:17.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:17 smithi007 ceph-mon[32184]: pgmap v394: 97 pgs: 97 active+clean; 39 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.8 MiB/s wr, 426 op/s 2024-03-23T13:21:19.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:19 smithi130 ceph-mon[36225]: pgmap v395: 97 pgs: 97 active+clean; 43 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.8 MiB/s wr, 353 op/s 2024-03-23T13:21:19.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:19 smithi155 ceph-mon[36157]: pgmap v395: 97 pgs: 97 active+clean; 43 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.8 MiB/s wr, 353 op/s 2024-03-23T13:21:19.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:19 smithi007 ceph-mon[32184]: pgmap v395: 97 pgs: 97 active+clean; 43 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.8 MiB/s wr, 353 op/s 2024-03-23T13:21:21.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:21 smithi130 ceph-mon[36225]: pgmap v396: 97 pgs: 97 active+clean; 43 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.6 MiB/s wr, 314 op/s 2024-03-23T13:21:21.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:21 smithi155 ceph-mon[36157]: pgmap v396: 97 pgs: 97 active+clean; 43 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.6 MiB/s wr, 314 op/s 2024-03-23T13:21:21.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:21 smithi007 ceph-mon[32184]: pgmap v396: 97 pgs: 97 active+clean; 43 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.6 MiB/s wr, 314 op/s 2024-03-23T13:21:22.037 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: executing libtool commands 2024-03-23T13:21:22.121 INFO:tasks.workunit.client.0.smithi007.stdout:Building include 2024-03-23T13:21:22.121 INFO:tasks.workunit.client.0.smithi007.stdout:Building lib 2024-03-23T13:21:22.122 INFO:tasks.workunit.client.0.smithi007.stdout:Building tests 2024-03-23T13:21:22.130 INFO:tasks.workunit.client.0.smithi007.stdout: [LTDEP] 2024-03-23T13:21:22.312 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] dataascii.lo 2024-03-23T13:21:22.320 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] databin.lo 2024-03-23T13:21:22.320 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] datapid.lo 2024-03-23T13:21:22.320 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] file_lock.lo 2024-03-23T13:21:22.459 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] forker.lo 2024-03-23T13:21:22.467 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] pattern.lo 2024-03-23T13:21:22.496 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] random_range.lo 2024-03-23T13:21:22.496 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] open_flags.lo 2024-03-23T13:21:22.623 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] string_to_tokens.lo 2024-03-23T13:21:22.634 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] str_to_bytes.lo 2024-03-23T13:21:22.669 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] tlibio.lo 2024-03-23T13:21:22.759 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] write_log.lo 2024-03-23T13:21:22.768 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] random.lo 2024-03-23T13:21:23.032 INFO:tasks.workunit.client.0.smithi007.stdout: [LD] 2024-03-23T13:21:23.143 INFO:tasks.workunit.client.0.smithi007.stdout:Building ltp 2024-03-23T13:21:23.143 INFO:tasks.workunit.client.0.smithi007.stdout:Building src 2024-03-23T13:21:23.143 INFO:tasks.workunit.client.0.smithi007.stdout:Building m4 2024-03-23T13:21:23.143 INFO:tasks.workunit.client.0.smithi007.stdout:Building common 2024-03-23T13:21:23.153 INFO:tasks.workunit.client.0.smithi007.stdout: [DEP] 2024-03-23T13:21:23.168 INFO:tasks.workunit.client.0.smithi007.stdout: [DEP] 2024-03-23T13:21:23.384 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] doio 2024-03-23T13:21:23.384 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] fsstress 2024-03-23T13:21:23.384 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] fsx 2024-03-23T13:21:23.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:23 smithi130 ceph-mon[36225]: pgmap v397: 97 pgs: 97 active+clean; 43 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.7 MiB/s wr, 304 op/s 2024-03-23T13:21:23.533 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c: In function 'attr_list_path': 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~ 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr:In file included from ../src/global.h:25, 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr: from fsstress.c:12: 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~ 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c: In function 'attr_remove_path': 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-03-23T13:21:23.534 INFO:tasks.workunit.client.0.smithi007.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~ 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr:In file included from ../src/global.h:25, 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr: from fsstress.c:12: 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~ 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c: In function 'attr_set_path': 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~ 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr:In file included from ../src/global.h:25, 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr: from fsstress.c:12: 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-03-23T13:21:23.535 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~ 2024-03-23T13:21:23.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:23 smithi155 ceph-mon[36157]: pgmap v397: 97 pgs: 97 active+clean; 43 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.7 MiB/s wr, 304 op/s 2024-03-23T13:21:23.545 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c: In function 'allocsp_f': 2024-03-23T13:21:23.545 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-03-23T13:21:23.545 INFO:tasks.workunit.client.0.smithi007.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-03-23T13:21:23.545 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~ 2024-03-23T13:21:23.545 INFO:tasks.workunit.client.0.smithi007.stderr: | XFS_IOC_RESVSP64 2024-03-23T13:21:23.545 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-03-23T13:21:23.562 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c: In function 'freesp_f': 2024-03-23T13:21:23.562 INFO:tasks.workunit.client.0.smithi007.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-03-23T13:21:23.562 INFO:tasks.workunit.client.0.smithi007.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-03-23T13:21:23.563 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~ 2024-03-23T13:21:23.563 INFO:tasks.workunit.client.0.smithi007.stderr: | XFS_IOC_RESVSP64 2024-03-23T13:21:23.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:23 smithi007 ceph-mon[32184]: pgmap v397: 97 pgs: 97 active+clean; 43 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.7 MiB/s wr, 304 op/s 2024-03-23T13:21:23.595 INFO:tasks.workunit.client.0.smithi007.stderr:gmake[3]: *** [Makefile:47: fsstress] Error 1 2024-03-23T13:21:23.595 INFO:tasks.workunit.client.0.smithi007.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-03-23T13:21:23.977 INFO:tasks.workunit.client.0.smithi007.stderr:doio.c: In function 'parse_cmdline': 2024-03-23T13:21:23.978 INFO:tasks.workunit.client.0.smithi007.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-03-23T13:21:23.978 INFO:tasks.workunit.client.0.smithi007.stderr: 3113 | fprintf(stderr, 2024-03-23T13:21:23.978 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~ 2024-03-23T13:21:23.978 INFO:tasks.workunit.client.0.smithi007.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-03-23T13:21:23.978 INFO:tasks.workunit.client.0.smithi007.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-23T13:21:23.978 INFO:tasks.workunit.client.0.smithi007.stderr: 3115 | Prog, TagName, tok); 2024-03-23T13:21:23.978 INFO:tasks.workunit.client.0.smithi007.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-03-23T13:21:24.002 INFO:tasks.workunit.client.0.smithi007.stderr:fsx.c: In function 'main': 2024-03-23T13:21:24.003 INFO:tasks.workunit.client.0.smithi007.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-03-23T13:21:24.003 INFO:tasks.workunit.client.0.smithi007.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-03-23T13:21:24.003 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-23T13:21:24.003 INFO:tasks.workunit.client.0.smithi007.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-03-23T13:21:24.003 INFO:tasks.workunit.client.0.smithi007.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-03-23T13:21:24.003 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-23T13:21:24.102 INFO:tasks.workunit.client.0.smithi007.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-03-23T13:21:24.102 INFO:tasks.workunit.client.0.smithi007.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-03-23T13:21:24.624 INFO:tasks.workunit.client.0.smithi007.stdout: [DEP] 2024-03-23T13:21:24.625 INFO:tasks.workunit.client.0.smithi007.stdout: [DEP] 2024-03-23T13:21:24.625 INFO:tasks.workunit.client.0.smithi007.stdout: [DEP] 2024-03-23T13:21:24.840 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] dirstress 2024-03-23T13:21:24.842 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] fill 2024-03-23T13:21:24.842 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] fill2 2024-03-23T13:21:24.843 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] getpagesize 2024-03-23T13:21:25.074 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] lstat64 2024-03-23T13:21:25.074 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] holes 2024-03-23T13:21:25.097 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] nametest 2024-03-23T13:21:25.106 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] permname 2024-03-23T13:21:25.271 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] randholes 2024-03-23T13:21:25.277 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] runas 2024-03-23T13:21:25.309 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] truncfile 2024-03-23T13:21:25.349 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] usemem 2024-03-23T13:21:25.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:25 smithi130 ceph-mon[36225]: pgmap v398: 97 pgs: 97 active+clean; 43 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 1.3 MiB/s wr, 241 op/s 2024-03-23T13:21:25.450 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] mmapcat 2024-03-23T13:21:25.494 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] append_reader 2024-03-23T13:21:25.501 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] append_writer 2024-03-23T13:21:25.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:25 smithi155 ceph-mon[36157]: pgmap v398: 97 pgs: 97 active+clean; 43 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 1.3 MiB/s wr, 241 op/s 2024-03-23T13:21:25.541 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] dirperf 2024-03-23T13:21:25.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:25 smithi007 ceph-mon[32184]: pgmap v398: 97 pgs: 97 active+clean; 43 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 1.3 MiB/s wr, 241 op/s 2024-03-23T13:21:25.609 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] metaperf 2024-03-23T13:21:25.651 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] devzero 2024-03-23T13:21:25.657 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] feature 2024-03-23T13:21:25.791 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] alloc 2024-03-23T13:21:25.832 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] fault 2024-03-23T13:21:25.853 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] fstest 2024-03-23T13:21:25.898 INFO:tasks.workunit.client.0.smithi007.stdout: [CC] t_access_root 2024-03-23T13:21:25.934 INFO:tasks.workunit.client.0.smithi007.stderr:alloc.c: In function 'main': 2024-03-23T13:21:25.934 INFO:tasks.workunit.client.0.smithi007.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-03-23T13:21:25.935 INFO:tasks.workunit.client.0.smithi007.stderr: 154 | bozo! 2024-03-23T13:21:25.935 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~ 2024-03-23T13:21:25.935 INFO:tasks.workunit.client.0.smithi007.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-03-23T13:21:25.935 INFO:tasks.workunit.client.0.smithi007.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-03-23T13:21:25.935 INFO:tasks.workunit.client.0.smithi007.stderr: 154 | bozo! 2024-03-23T13:21:25.935 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-03-23T13:21:25.935 INFO:tasks.workunit.client.0.smithi007.stderr: | ; 2024-03-23T13:21:25.935 INFO:tasks.workunit.client.0.smithi007.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-03-23T13:21:25.936 INFO:tasks.workunit.client.0.smithi007.stderr: 184 | rflag = 1; 2024-03-23T13:21:25.936 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~ 2024-03-23T13:21:25.936 INFO:tasks.workunit.client.0.smithi007.stderr: | nflag 2024-03-23T13:21:25.936 INFO:tasks.workunit.client.0.smithi007.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-03-23T13:21:25.936 INFO:tasks.workunit.client.0.smithi007.stderr: 320 | bozo! 2024-03-23T13:21:25.936 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-03-23T13:21:25.936 INFO:tasks.workunit.client.0.smithi007.stderr: | ; 2024-03-23T13:21:25.936 INFO:tasks.workunit.client.0.smithi007.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-03-23T13:21:25.936 INFO:tasks.workunit.client.0.smithi007.stderr: 128 | struct flock64 f; 2024-03-23T13:21:25.936 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-03-23T13:21:25.939 INFO:tasks.workunit.client.0.smithi007.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2024-03-23T13:21:25.940 INFO:tasks.workunit.client.0.smithi007.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-03-23T13:21:26.011 INFO:tasks.workunit.client.0.smithi007.stderr:fstest.c: In function 'run_child': 2024-03-23T13:21:26.011 INFO:tasks.workunit.client.0.smithi007.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-03-23T13:21:26.011 INFO:tasks.workunit.client.0.smithi007.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-23T13:21:26.011 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~ 2024-03-23T13:21:26.011 INFO:tasks.workunit.client.0.smithi007.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-03-23T13:21:26.011 INFO:tasks.workunit.client.0.smithi007.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~ 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~ 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~ 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-23T13:21:26.012 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-23T13:21:26.083 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-23T13:21:26.083 INFO:tasks.workunit.client.0.smithi007.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-03-23T13:21:26.084 INFO:tasks.workunit.client.0.smithi007.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-03-23T13:21:26.084 INFO:tasks.workunit.client.0.smithi007.stderr:make: *** [Makefile:51: default] Error 2 2024-03-23T13:21:26.084 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-03-23T13:21:26.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-23T13:21:26.376 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi007 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-03-23T13:21:26.606 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=bdd31bdb3e27436abefdad26345b3dd4 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi007 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-03-23T13:21:26.609 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-23T13:21:26.619 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi007.front.sepia.ceph.com 2024-03-23T13:21:26.632 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-23T13:21:26.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:21:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.776+0000 7fb7da3c9640 1 -- 172.21.15.7:0/1172302151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d4102200 msgr2=0x7fb7d4108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.776+0000 7fb7da3c9640 1 --2- 172.21.15.7:0/1172302151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d4102200 0x7fb7d4108a90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009ca0 tx=0x7fb7cc02c0f0 comp rx=0 tx=0).stop 2024-03-23T13:21:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 -- 172.21.15.7:0/1172302151 shutdown_connections 2024-03-23T13:21:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 --2- 172.21.15.7:0/1172302151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d4102200 0x7fb7d4108a90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 --2- 172.21.15.7:0/1172302151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d4101840 0x7fb7d4101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 --2- 172.21.15.7:0/1172302151 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d4100650 0x7fb7d4100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 -- 172.21.15.7:0/1172302151 >> 172.21.15.7:0/1172302151 conn(0x7fb7d40fbbc0 msgr2=0x7fb7d40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 -- 172.21.15.7:0/1172302151 shutdown_connections 2024-03-23T13:21:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 -- 172.21.15.7:0/1172302151 wait complete. 2024-03-23T13:21:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 Processor -- start 2024-03-23T13:21:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 -- start start 2024-03-23T13:21:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d4100650 0x7fb7d406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d4101840 0x7fb7d419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d4102200 0x7fb7d419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d419f300 con 0x7fb7d4102200 2024-03-23T13:21:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d419f470 con 0x7fb7d4101840 2024-03-23T13:21:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.777+0000 7fb7da3c9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d419f630 con 0x7fb7d4100650 2024-03-23T13:21:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d8966640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d4102200 0x7fb7d419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d3fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d4100650 0x7fb7d406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d8966640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d4102200 0x7fb7d419eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44790/0 (socket says 172.21.15.7:44790) 2024-03-23T13:21:26.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d3fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d4100650 0x7fb7d406a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.7:42798/0 (socket says 172.21.15.7:42798) 2024-03-23T13:21:26.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d8966640 1 -- 172.21.15.7:0/3636583834 learned_addr learned my addr 172.21.15.7:0/3636583834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:26.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d37fe640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d4101840 0x7fb7d419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:26.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d8966640 1 -- 172.21.15.7:0/3636583834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d4100650 msgr2=0x7fb7d406a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:26.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d8966640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d4100650 0x7fb7d406a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:26.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d8966640 1 -- 172.21.15.7:0/3636583834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d4101840 msgr2=0x7fb7d419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:26.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d8966640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d4101840 0x7fb7d419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:26.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d8966640 1 -- 172.21.15.7:0/3636583834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc009900 con 0x7fb7d4102200 2024-03-23T13:21:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d8966640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d4102200 0x7fb7d419eb20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc037040 tx=0x7fb7cc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d17fa640 1 -- 172.21.15.7:0/3636583834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc0329b0 con 0x7fb7d4102200 2024-03-23T13:21:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d17fa640 1 -- 172.21.15.7:0/3636583834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc003e50 con 0x7fb7d4102200 2024-03-23T13:21:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7da3c9640 1 -- 172.21.15.7:0/3636583834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d419f850 con 0x7fb7d4102200 2024-03-23T13:21:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7d17fa640 1 -- 172.21.15.7:0/3636583834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc0048a0 con 0x7fb7d4102200 2024-03-23T13:21:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.778+0000 7fb7da3c9640 1 -- 172.21.15.7:0/3636583834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d419fd50 con 0x7fb7d4102200 2024-03-23T13:21:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.779+0000 7fb7da3c9640 1 -- 172.21.15.7:0/3636583834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d4107910 con 0x7fb7d4102200 2024-03-23T13:21:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.780+0000 7fb7d17fa640 1 -- 172.21.15.7:0/3636583834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fb7cc00a040 con 0x7fb7d4102200 2024-03-23T13:21:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.781+0000 7fb7d17fa640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb7bc0a2140 0x7fb7bc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.781+0000 7fb7d3fff640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb7bc0a2140 0x7fb7bc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.781+0000 7fb7d17fa640 1 -- 172.21.15.7:0/3636583834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb7cc0f0d80 con 0x7fb7d4102200 2024-03-23T13:21:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.782+0000 7fb7d3fff640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb7bc0a2140 0x7fb7bc0a4600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb7d41016a0 tx=0x7fb7c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.785+0000 7fb7d17fa640 1 -- 172.21.15.7:0/3636583834 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb7cc0b9df0 con 0x7fb7d4102200 2024-03-23T13:21:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.970+0000 7fb7da3c9640 1 -- 172.21.15.7:0/3636583834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7d410c260 con 0x7fb7d4102200 2024-03-23T13:21:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.971+0000 7fb7d17fa640 1 -- 172.21.15.7:0/3636583834 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22558 (secure 0 0 0) 0x7fb7cc034e60 con 0x7fb7d4102200 2024-03-23T13:21:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:21:26.974 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:18:43.667397+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:21:26.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.973+0000 7fb7da3c9640 1 -- 172.21.15.7:0/3636583834 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb7bc0a2140 msgr2=0x7fb7bc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:26.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.973+0000 7fb7da3c9640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb7bc0a2140 0x7fb7bc0a4600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb7d41016a0 tx=0x7fb7c4005f90 comp rx=0 tx=0).stop 2024-03-23T13:21:26.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.973+0000 7fb7da3c9640 1 -- 172.21.15.7:0/3636583834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d4102200 msgr2=0x7fb7d419eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:26.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.973+0000 7fb7da3c9640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d4102200 0x7fb7d419eb20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc037040 tx=0x7fb7cc009360 comp rx=0 tx=0).stop 2024-03-23T13:21:26.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.974+0000 7fb7da3c9640 1 -- 172.21.15.7:0/3636583834 shutdown_connections 2024-03-23T13:21:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.974+0000 7fb7da3c9640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb7bc0a2140 0x7fb7bc0a4600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.974+0000 7fb7da3c9640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d4102200 0x7fb7d419eb20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.974+0000 7fb7da3c9640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7d4101840 0x7fb7d419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.974+0000 7fb7da3c9640 1 --2- 172.21.15.7:0/3636583834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d4100650 0x7fb7d406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.974+0000 7fb7da3c9640 1 -- 172.21.15.7:0/3636583834 >> 172.21.15.7:0/3636583834 conn(0x7fb7d40fbbc0 msgr2=0x7fb7d4104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.974+0000 7fb7da3c9640 1 -- 172.21.15.7:0/3636583834 shutdown_connections 2024-03-23T13:21:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:26.974+0000 7fb7da3c9640 1 -- 172.21.15.7:0/3636583834 wait complete. 2024-03-23T13:21:26.988 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T13:13:08.186719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T13:17:38.843887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T13:17:39.788489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T13:21:26.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T13:21:27.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.092+0000 7facf6a81640 1 -- 172.21.15.7:0/20263665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf0101710 msgr2=0x7facf0101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.092+0000 7facf6a81640 1 --2- 172.21.15.7:0/20263665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf0101710 0x7facf0101b30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7face4009900 tx=0x7face402bc30 comp rx=0 tx=0).stop 2024-03-23T13:21:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.092+0000 7facf6a81640 1 -- 172.21.15.7:0/20263665 shutdown_connections 2024-03-23T13:21:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.092+0000 7facf6a81640 1 --2- 172.21.15.7:0/20263665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf01032c0 0x7facf0109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.092+0000 7facf6a81640 1 --2- 172.21.15.7:0/20263665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0102900 0x7facf0102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.092+0000 7facf6a81640 1 --2- 172.21.15.7:0/20263665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf0101710 0x7facf0101b30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.092+0000 7facf6a81640 1 -- 172.21.15.7:0/20263665 >> 172.21.15.7:0/20263665 conn(0x7facf00fcc80 msgr2=0x7facf00ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.092+0000 7facf6a81640 1 -- 172.21.15.7:0/20263665 shutdown_connections 2024-03-23T13:21:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.092+0000 7facf6a81640 1 -- 172.21.15.7:0/20263665 wait complete. 2024-03-23T13:21:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf6a81640 1 Processor -- start 2024-03-23T13:21:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf6a81640 1 -- start start 2024-03-23T13:21:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf6a81640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf0101710 0x7facf01999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf6a81640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0102900 0x7facf0199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf6a81640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf01032c0 0x7facf01a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf6a81640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7facf01a14e0 con 0x7facf01032c0 2024-03-23T13:21:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf6a81640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7facf01a1650 con 0x7facf0101710 2024-03-23T13:21:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf6a81640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7facf01a1810 con 0x7facf0102900 2024-03-23T13:21:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf501e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf01032c0 0x7facf01a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf481d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf0101710 0x7facf01999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf481d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf0101710 0x7facf01999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.7:52494/0 (socket says 172.21.15.7:52494) 2024-03-23T13:21:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf501e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf01032c0 0x7facf01a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44800/0 (socket says 172.21.15.7:44800) 2024-03-23T13:21:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf481d640 1 -- 172.21.15.7:0/640611628 learned_addr learned my addr 172.21.15.7:0/640611628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7faceffff640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0102900 0x7facf0199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf501e640 1 -- 172.21.15.7:0/640611628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0102900 msgr2=0x7facf0199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf501e640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0102900 0x7facf0199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf501e640 1 -- 172.21.15.7:0/640611628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf0101710 msgr2=0x7facf01999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf501e640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf0101710 0x7facf01999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf501e640 1 -- 172.21.15.7:0/640611628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face4009560 con 0x7facf01032c0 2024-03-23T13:21:27.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf481d640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf0101710 0x7facf01999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:21:27.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.093+0000 7facf501e640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf01032c0 0x7facf01a0fa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7face8009fd0 tx=0x7face800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:27.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.094+0000 7facedffb640 1 -- 172.21.15.7:0/640611628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face800ce20 con 0x7facf01032c0 2024-03-23T13:21:27.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.094+0000 7facedffb640 1 -- 172.21.15.7:0/640611628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face8005020 con 0x7facf01032c0 2024-03-23T13:21:27.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.094+0000 7facf6a81640 1 -- 172.21.15.7:0/640611628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf01a1af0 con 0x7facf01032c0 2024-03-23T13:21:27.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.094+0000 7facedffb640 1 -- 172.21.15.7:0/640611628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8010690 con 0x7facf01032c0 2024-03-23T13:21:27.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.094+0000 7facf6a81640 1 -- 172.21.15.7:0/640611628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf01a1fb0 con 0x7facf01032c0 2024-03-23T13:21:27.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.095+0000 7facf6a81640 1 -- 172.21.15.7:0/640611628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf010d110 con 0x7facf01032c0 2024-03-23T13:21:27.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.096+0000 7facedffb640 1 -- 172.21.15.7:0/640611628 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7face8005500 con 0x7facf01032c0 2024-03-23T13:21:27.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.096+0000 7facedffb640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7face00a20c0 0x7face00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.096+0000 7facf481d640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7face00a20c0 0x7face00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.096+0000 7facedffb640 1 -- 172.21.15.7:0/640611628 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7face8014070 con 0x7facf01032c0 2024-03-23T13:21:27.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.097+0000 7facf481d640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7face00a20c0 0x7face00a4580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7face402c840 tx=0x7face402cd10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:27.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.101+0000 7facedffb640 1 -- 172.21.15.7:0/640611628 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7face80973e0 con 0x7facf01032c0 2024-03-23T13:21:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.283+0000 7facf6a81640 1 -- 172.21.15.7:0/640611628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7facf01a23d0 con 0x7facf01032c0 2024-03-23T13:21:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.283+0000 7facedffb640 1 -- 172.21.15.7:0/640611628 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7face8005190 con 0x7facf01032c0 2024-03-23T13:21:27.285 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-23T13:21:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 -- 172.21.15.7:0/640611628 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7face00a20c0 msgr2=0x7face00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7face00a20c0 0x7face00a4580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7face402c840 tx=0x7face402cd10 comp rx=0 tx=0).stop 2024-03-23T13:21:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 -- 172.21.15.7:0/640611628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf01032c0 msgr2=0x7facf01a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf01032c0 0x7facf01a0fa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7face8009fd0 tx=0x7face800ef10 comp rx=0 tx=0).stop 2024-03-23T13:21:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 -- 172.21.15.7:0/640611628 shutdown_connections 2024-03-23T13:21:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7face00a20c0 0x7face00a4580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf01032c0 0x7facf01a0fa0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0102900 0x7facf0199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 --2- 172.21.15.7:0/640611628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7facf0101710 0x7facf01999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 -- 172.21.15.7:0/640611628 >> 172.21.15.7:0/640611628 conn(0x7facf00fcc80 msgr2=0x7facf0073940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 -- 172.21.15.7:0/640611628 shutdown_connections 2024-03-23T13:21:27.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.285+0000 7facf6a81640 1 -- 172.21.15.7:0/640611628 wait complete. 2024-03-23T13:21:27.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:27 smithi007 ceph-mon[32184]: pgmap v399: 97 pgs: 97 active+clean; 43 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 967 KiB/s wr, 186 op/s 2024-03-23T13:21:27.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:27 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3636583834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:21:27.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T13:21:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 -- 172.21.15.7:0/2076836471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe574101c40 msgr2=0x7fe574102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 --2- 172.21.15.7:0/2076836471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe574101c40 0x7fe574102060 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe570009900 tx=0x7fe57002bc30 comp rx=0 tx=0).stop 2024-03-23T13:21:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 -- 172.21.15.7:0/2076836471 shutdown_connections 2024-03-23T13:21:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 --2- 172.21.15.7:0/2076836471 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe574101250 0x7fe574108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 --2- 172.21.15.7:0/2076836471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe574100890 0x7fe574100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 --2- 172.21.15.7:0/2076836471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe574101c40 0x7fe574102060 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 -- 172.21.15.7:0/2076836471 >> 172.21.15.7:0/2076836471 conn(0x7fe5740fb340 msgr2=0x7fe5740fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 -- 172.21.15.7:0/2076836471 shutdown_connections 2024-03-23T13:21:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 -- 172.21.15.7:0/2076836471 wait complete. 2024-03-23T13:21:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 Processor -- start 2024-03-23T13:21:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.403+0000 7fe57b895640 1 -- start start 2024-03-23T13:21:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe57b895640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe574100890 0x7fe574197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe57b895640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe574101250 0x7fe574197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe57b895640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe574101c40 0x7fe57419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe57b895640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe57419f320 con 0x7fe574101250 2024-03-23T13:21:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe57b895640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe57419f490 con 0x7fe574100890 2024-03-23T13:21:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe57b895640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe57419f650 con 0x7fe574101c40 2024-03-23T13:21:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe578e30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe574101250 0x7fe574197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe578e30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe574101250 0x7fe574197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44816/0 (socket says 172.21.15.7:44816) 2024-03-23T13:21:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe578e30640 1 -- 172.21.15.7:0/3182867207 learned_addr learned my addr 172.21.15.7:0/3182867207 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe579631640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe574100890 0x7fe574197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe579e32640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe574101c40 0x7fe57419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe578e30640 1 -- 172.21.15.7:0/3182867207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe574101c40 msgr2=0x7fe57419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe578e30640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe574101c40 0x7fe57419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe578e30640 1 -- 172.21.15.7:0/3182867207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe574100890 msgr2=0x7fe574197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe578e30640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe574100890 0x7fe574197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe578e30640 1 -- 172.21.15.7:0/3182867207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe570009560 con 0x7fe574101250 2024-03-23T13:21:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe578e30640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe574101250 0x7fe574197d70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe56400e9c0 tx=0x7fe56400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe56a7fc640 1 -- 172.21.15.7:0/3182867207 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640096e0 con 0x7fe574101250 2024-03-23T13:21:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe56a7fc640 1 -- 172.21.15.7:0/3182867207 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56400cd80 con 0x7fe574101250 2024-03-23T13:21:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe57b895640 1 -- 172.21.15.7:0/3182867207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57419f930 con 0x7fe574101250 2024-03-23T13:21:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe56a7fc640 1 -- 172.21.15.7:0/3182867207 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564010430 con 0x7fe574101250 2024-03-23T13:21:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.404+0000 7fe57b895640 1 -- 172.21.15.7:0/3182867207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57419fdf0 con 0x7fe574101250 2024-03-23T13:21:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.405+0000 7fe57b895640 1 -- 172.21.15.7:0/3182867207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe574107a50 con 0x7fe574101250 2024-03-23T13:21:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.407+0000 7fe56a7fc640 1 -- 172.21.15.7:0/3182867207 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fe564003680 con 0x7fe574101250 2024-03-23T13:21:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.407+0000 7fe56a7fc640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5600a20c0 0x7fe5600a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.408+0000 7fe579631640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5600a20c0 0x7fe5600a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.408+0000 7fe56a7fc640 1 -- 172.21.15.7:0/3182867207 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe564014070 con 0x7fe574101250 2024-03-23T13:21:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.408+0000 7fe579631640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5600a20c0 0x7fe5600a4580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe570005ab0 tx=0x7fe570005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:27.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.411+0000 7fe56a7fc640 1 -- 172.21.15.7:0/3182867207 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fe5640970d0 con 0x7fe574101250 2024-03-23T13:21:27.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:27 smithi130 ceph-mon[36225]: pgmap v399: 97 pgs: 97 active+clean; 43 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 967 KiB/s wr, 186 op/s 2024-03-23T13:21:27.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:27 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3636583834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:21:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:27 smithi155 ceph-mon[36157]: pgmap v399: 97 pgs: 97 active+clean; 43 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 967 KiB/s wr, 186 op/s 2024-03-23T13:21:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:27 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3636583834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:21:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.593+0000 7fe57b895640 1 -- 172.21.15.7:0/3182867207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe57410d3f0 con 0x7fe574101250 2024-03-23T13:21:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.593+0000 7fe56a7fc640 1 -- 172.21.15.7:0/3182867207 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fe56409a790 con 0x7fe574101250 2024-03-23T13:21:27.596 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-23T13:21:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 -- 172.21.15.7:0/3182867207 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5600a20c0 msgr2=0x7fe5600a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5600a20c0 0x7fe5600a4580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe570005ab0 tx=0x7fe570005fd0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 -- 172.21.15.7:0/3182867207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe574101250 msgr2=0x7fe574197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe574101250 0x7fe574197d70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe56400e9c0 tx=0x7fe56400ee80 comp rx=0 tx=0).stop 2024-03-23T13:21:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 -- 172.21.15.7:0/3182867207 shutdown_connections 2024-03-23T13:21:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fe5600a20c0 0x7fe5600a4580 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe574101c40 0x7fe57419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe574101250 0x7fe574197d70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 --2- 172.21.15.7:0/3182867207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe574100890 0x7fe574197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 -- 172.21.15.7:0/3182867207 >> 172.21.15.7:0/3182867207 conn(0x7fe5740fb340 msgr2=0x7fe574104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 -- 172.21.15.7:0/3182867207 shutdown_connections 2024-03-23T13:21:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.595+0000 7fe57b895640 1 -- 172.21.15.7:0/3182867207 wait complete. 2024-03-23T13:21:27.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T13:21:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.713+0000 7fef5f634640 1 -- 172.21.15.7:0/2444694958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef58101800 msgr2=0x7fef58101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.713+0000 7fef5f634640 1 --2- 172.21.15.7:0/2444694958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef58101800 0x7fef58101c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009900 tx=0x7fef4c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:21:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.713+0000 7fef5f634640 1 -- 172.21.15.7:0/2444694958 shutdown_connections 2024-03-23T13:21:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.713+0000 7fef5f634640 1 --2- 172.21.15.7:0/2444694958 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef581021c0 0x7fef58108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.713+0000 7fef5f634640 1 --2- 172.21.15.7:0/2444694958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef58101800 0x7fef58101c80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.713+0000 7fef5f634640 1 --2- 172.21.15.7:0/2444694958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef58100610 0x7fef58100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.713+0000 7fef5f634640 1 -- 172.21.15.7:0/2444694958 >> 172.21.15.7:0/2444694958 conn(0x7fef580fbb80 msgr2=0x7fef580fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.713+0000 7fef5f634640 1 -- 172.21.15.7:0/2444694958 shutdown_connections 2024-03-23T13:21:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.713+0000 7fef5f634640 1 -- 172.21.15.7:0/2444694958 wait complete. 2024-03-23T13:21:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5f634640 1 Processor -- start 2024-03-23T13:21:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5f634640 1 -- start start 2024-03-23T13:21:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5f634640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef58100610 0x7fef581045a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5f634640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef58101800 0x7fef58102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5f634640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef581021c0 0x7fef58103120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5f634640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fef58103880 con 0x7fef58100610 2024-03-23T13:21:27.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5f634640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fef581039f0 con 0x7fef581021c0 2024-03-23T13:21:27.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5f634640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef581a1850 con 0x7fef58101800 2024-03-23T13:21:27.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5d3d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef58100610 0x7fef581045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5d3d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef58100610 0x7fef581045a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44834/0 (socket says 172.21.15.7:44834) 2024-03-23T13:21:27.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5d3d0640 1 -- 172.21.15.7:0/255391585 learned_addr learned my addr 172.21.15.7:0/255391585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:27.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5cbcf640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef58101800 0x7fef58102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5dbd1640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef581021c0 0x7fef58103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5d3d0640 1 -- 172.21.15.7:0/255391585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef58101800 msgr2=0x7fef58102be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5d3d0640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef58101800 0x7fef58102be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5d3d0640 1 -- 172.21.15.7:0/255391585 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef581021c0 msgr2=0x7fef58103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5d3d0640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef581021c0 0x7fef58103120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5d3d0640 1 -- 172.21.15.7:0/255391585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c009560 con 0x7fef58100610 2024-03-23T13:21:27.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5d3d0640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef58100610 0x7fef581045a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fef5000e9f0 tx=0x7fef5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:27.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef567fc640 1 -- 172.21.15.7:0/255391585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef500096e0 con 0x7fef58100610 2024-03-23T13:21:27.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef5f634640 1 -- 172.21.15.7:0/255391585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef581a1b00 con 0x7fef58100610 2024-03-23T13:21:27.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.714+0000 7fef567fc640 1 -- 172.21.15.7:0/255391585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef5000cd80 con 0x7fef58100610 2024-03-23T13:21:27.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.715+0000 7fef567fc640 1 -- 172.21.15.7:0/255391585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50010430 con 0x7fef58100610 2024-03-23T13:21:27.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.715+0000 7fef5f634640 1 -- 172.21.15.7:0/255391585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef581a1f90 con 0x7fef58100610 2024-03-23T13:21:27.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.716+0000 7fef5f634640 1 -- 172.21.15.7:0/255391585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5810c020 con 0x7fef58100610 2024-03-23T13:21:27.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.717+0000 7fef567fc640 1 -- 172.21.15.7:0/255391585 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fef50003680 con 0x7fef58100610 2024-03-23T13:21:27.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.717+0000 7fef567fc640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef400a20c0 0x7fef400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:27.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.717+0000 7fef5cbcf640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef400a20c0 0x7fef400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:27.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.717+0000 7fef567fc640 1 -- 172.21.15.7:0/255391585 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fef50014070 con 0x7fef58100610 2024-03-23T13:21:27.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.718+0000 7fef5cbcf640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef400a20c0 0x7fef400a4580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009230 tx=0x7fef4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:27.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.721+0000 7fef567fc640 1 -- 172.21.15.7:0/255391585 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fef50097050 con 0x7fef58100610 2024-03-23T13:21:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.903+0000 7fef5f634640 1 -- 172.21.15.7:0/255391585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef581a2120 con 0x7fef58100610 2024-03-23T13:21:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.903+0000 7fef567fc640 1 -- 172.21.15.7:0/255391585 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fef5009a710 con 0x7fef58100610 2024-03-23T13:21:27.905 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-23T13:21:27.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 -- 172.21.15.7:0/255391585 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef400a20c0 msgr2=0x7fef400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef400a20c0 0x7fef400a4580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009230 tx=0x7fef4c005fd0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 -- 172.21.15.7:0/255391585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef58100610 msgr2=0x7fef581045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef58100610 0x7fef581045a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fef5000e9f0 tx=0x7fef5000eeb0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 -- 172.21.15.7:0/255391585 shutdown_connections 2024-03-23T13:21:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef400a20c0 0x7fef400a4580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef581021c0 0x7fef58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef58101800 0x7fef58102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 --2- 172.21.15.7:0/255391585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef58100610 0x7fef581045a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 -- 172.21.15.7:0/255391585 >> 172.21.15.7:0/255391585 conn(0x7fef580fbb80 msgr2=0x7fef580fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 -- 172.21.15.7:0/255391585 shutdown_connections 2024-03-23T13:21:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:27.905+0000 7fef5f634640 1 -- 172.21.15.7:0/255391585 wait complete. 2024-03-23T13:21:27.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-23T13:21:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.019+0000 7fc304747640 1 -- 172.21.15.7:0/1476602142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc101970 msgr2=0x7fc2fc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.019+0000 7fc304747640 1 --2- 172.21.15.7:0/1476602142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc101970 0x7fc2fc108200 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800adb0 tx=0x7fc2f802cff0 comp rx=0 tx=0).stop 2024-03-23T13:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.019+0000 7fc304747640 1 -- 172.21.15.7:0/1476602142 shutdown_connections 2024-03-23T13:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.019+0000 7fc304747640 1 --2- 172.21.15.7:0/1476602142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc101970 0x7fc2fc108200 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.019+0000 7fc304747640 1 --2- 172.21.15.7:0/1476602142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2fc100fb0 0x7fc2fc101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.019+0000 7fc304747640 1 --2- 172.21.15.7:0/1476602142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2fc0ffdc0 0x7fc2fc1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.019+0000 7fc304747640 1 -- 172.21.15.7:0/1476602142 >> 172.21.15.7:0/1476602142 conn(0x7fc2fc0fb330 msgr2=0x7fc2fc0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.019+0000 7fc304747640 1 -- 172.21.15.7:0/1476602142 shutdown_connections 2024-03-23T13:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.019+0000 7fc304747640 1 -- 172.21.15.7:0/1476602142 wait complete. 2024-03-23T13:21:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc304747640 1 Processor -- start 2024-03-23T13:21:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc304747640 1 -- start start 2024-03-23T13:21:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc304747640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2fc0ffdc0 0x7fc2fc103e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc304747640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc100fb0 0x7fc2fc102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc304747640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2fc101970 0x7fc2fc1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc304747640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc103100 con 0x7fc2fc100fb0 2024-03-23T13:21:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc304747640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc103270 con 0x7fc2fc0ffdc0 2024-03-23T13:21:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc304747640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1a1810 con 0x7fc2fc101970 2024-03-23T13:21:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc301ce2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc100fb0 0x7fc2fc102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc301ce2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc100fb0 0x7fc2fc102460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44854/0 (socket says 172.21.15.7:44854) 2024-03-23T13:21:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc301ce2640 1 -- 172.21.15.7:0/4135422947 learned_addr learned my addr 172.21.15.7:0/4135422947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:21:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc3024e3640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2fc0ffdc0 0x7fc2fc103e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.020+0000 7fc302ce4640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2fc101970 0x7fc2fc1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc301ce2640 1 -- 172.21.15.7:0/4135422947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2fc101970 msgr2=0x7fc2fc1029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc301ce2640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2fc101970 0x7fc2fc1029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc301ce2640 1 -- 172.21.15.7:0/4135422947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2fc0ffdc0 msgr2=0x7fc2fc103e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc301ce2640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2fc0ffdc0 0x7fc2fc103e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc301ce2640 1 -- 172.21.15.7:0/4135422947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f800aa10 con 0x7fc2fc100fb0 2024-03-23T13:21:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc301ce2640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc100fb0 0x7fc2fc102460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00ea20 tx=0x7fc2ec00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc2f37fe640 1 -- 172.21.15.7:0/4135422947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec009720 con 0x7fc2fc100fb0 2024-03-23T13:21:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc304747640 1 -- 172.21.15.7:0/4135422947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc1a1ac0 con 0x7fc2fc100fb0 2024-03-23T13:21:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc2f37fe640 1 -- 172.21.15.7:0/4135422947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ec00cd80 con 0x7fc2fc100fb0 2024-03-23T13:21:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc2f37fe640 1 -- 172.21.15.7:0/4135422947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec010490 con 0x7fc2fc100fb0 2024-03-23T13:21:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.021+0000 7fc304747640 1 -- 172.21.15.7:0/4135422947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc1a1fd0 con 0x7fc2fc100fb0 2024-03-23T13:21:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.022+0000 7fc304747640 1 -- 172.21.15.7:0/4135422947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2fc10af50 con 0x7fc2fc100fb0 2024-03-23T13:21:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.023+0000 7fc2f37fe640 1 -- 172.21.15.7:0/4135422947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fc2ec005020 con 0x7fc2fc100fb0 2024-03-23T13:21:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.024+0000 7fc2f37fe640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc2e80a20c0 0x7fc2e80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:21:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.024+0000 7fc3024e3640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc2e80a20c0 0x7fc2e80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:21:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.024+0000 7fc2f37fe640 1 -- 172.21.15.7:0/4135422947 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc2ec014070 con 0x7fc2fc100fb0 2024-03-23T13:21:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.024+0000 7fc3024e3640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc2e80a20c0 0x7fc2e80a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc100e10 tx=0x7fc2f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:21:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.028+0000 7fc2f37fe640 1 -- 172.21.15.7:0/4135422947 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc2ec097060 con 0x7fc2fc100fb0 2024-03-23T13:21:28.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.261+0000 7fc304747640 1 -- 172.21.15.7:0/4135422947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fc2fc1a2470 con 0x7fc2fc100fb0 2024-03-23T13:21:28.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.269+0000 7fc2f37fe640 1 -- 172.21.15.7:0/4135422947 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+736389 (secure 0 0 0) 0x7fc2ec09a720 con 0x7fc2fc100fb0 2024-03-23T13:21:28.282 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:21:28.282 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:21:28.282 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 18, 2024-03-23T13:21:28.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.282 INFO:teuthology.orchestra.run.smithi007.stdout: "active_gid": 14152, 2024-03-23T13:21:28.282 INFO:teuthology.orchestra.run.smithi007.stdout: "active_name": "x", 2024-03-23T13:21:28.283 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addrs": { 2024-03-23T13:21:28.283 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-23T13:21:28.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-23T13:21:28.283 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-03-23T13:21:28.283 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1535208045 2024-03-23T13:21:28.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-03-23T13:21:28.284 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-03-23T13:21:28.284 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1535208045 2024-03-23T13:21:28.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.284 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-23T13:21:28.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.284 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addr": "172.21.15.7:6801/1535208045", 2024-03-23T13:21:28.284 INFO:teuthology.orchestra.run.smithi007.stdout: "active_change": "2024-03-23T13:10:41.843766+0000", 2024-03-23T13:21:28.284 INFO:teuthology.orchestra.run.smithi007.stdout: "active_mgr_features": 4540701547738038271, 2024-03-23T13:21:28.285 INFO:teuthology.orchestra.run.smithi007.stdout: "available": true, 2024-03-23T13:21:28.285 INFO:teuthology.orchestra.run.smithi007.stdout: "standbys": [ 2024-03-23T13:21:28.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.285 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24109, 2024-03-23T13:21:28.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "z", 2024-03-23T13:21:28.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-03-23T13:21:28.285 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-03-23T13:21:28.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-03-23T13:21:28.286 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.286 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.286 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.286 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:28.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:28.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.287 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.287 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:28.287 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.287 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.287 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.287 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T13:21:28.287 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.288 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.288 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.289 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.289 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.289 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.289 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.289 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.289 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.290 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.290 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.290 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.291 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.291 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.291 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.291 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.292 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.292 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.292 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.295 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.295 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.295 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.295 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.295 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.296 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.296 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.296 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.296 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-03-23T13:21:28.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-03-23T13:21:28.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.297 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.297 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.297 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.297 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.297 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-03-23T13:21:28.297 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.298 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.298 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-03-23T13:21:28.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-03-23T13:21:28.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.298 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.298 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-03-23T13:21:28.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.299 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-03-23T13:21:28.299 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.299 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.300 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-03-23T13:21:28.300 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-03-23T13:21:28.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.300 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.300 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.300 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.300 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.300 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.301 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-03-23T13:21:28.301 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.301 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.301 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-03-23T13:21:28.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-03-23T13:21:28.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.302 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.302 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.302 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-03-23T13:21:28.302 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.303 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.303 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-03-23T13:21:28.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-03-23T13:21:28.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.303 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-03-23T13:21:28.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.304 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.304 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-03-23T13:21:28.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.304 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.304 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-03-23T13:21:28.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-03-23T13:21:28.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.305 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.305 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-03-23T13:21:28.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.306 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-03-23T13:21:28.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-03-23T13:21:28.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.307 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T13:21:28.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.308 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-03-23T13:21:28.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-03-23T13:21:28.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.308 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.308 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.309 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.309 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-03-23T13:21:28.309 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.309 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-03-23T13:21:28.310 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.310 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.310 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.310 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-03-23T13:21:28.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-03-23T13:21:28.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.311 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.311 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T13:21:28.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.312 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-03-23T13:21:28.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-03-23T13:21:28.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.312 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-03-23T13:21:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T13:21:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T13:21:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-03-23T13:21:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-03-23T13:21:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-23T13:21:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-23T13:21:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T13:21:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T13:21:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-03-23T13:21:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-03-23T13:21:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-03-23T13:21:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-03-23T13:21:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T13:21:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-03-23T13:21:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-03-23T13:21:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T13:21:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T13:21:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T13:21:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-03-23T13:21:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-03-23T13:21:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-03-23T13:21:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-03-23T13:21:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-03-23T13:21:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-03-23T13:21:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T13:21:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-03-23T13:21:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-03-23T13:21:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-03-23T13:21:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T13:21:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T13:21:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-03-23T13:21:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-03-23T13:21:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-23T13:21:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-23T13:21:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T13:21:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T13:21:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-03-23T13:21:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-03-23T13:21:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T13:21:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-23T13:21:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-23T13:21:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-03-23T13:21:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-03-23T13:21:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-03-23T13:21:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-03-23T13:21:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-03-23T13:21:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-03-23T13:21:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-03-23T13:21:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-03-23T13:21:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-03-23T13:21:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T13:21:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T13:21:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-03-23T13:21:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-03-23T13:21:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T13:21:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T13:21:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-03-23T13:21:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-03-23T13:21:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:28.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.343 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.343 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T13:21:28.343 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.343 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.344 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-03-23T13:21:28.344 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.344 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.344 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.344 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-03-23T13:21:28.344 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-03-23T13:21:28.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:28.345 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.345 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-23T13:21:28.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.345 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.345 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.345 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T13:21:28.345 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.346 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.346 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-03-23T13:21:28.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-03-23T13:21:28.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.346 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.346 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-03-23T13:21:28.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.347 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.347 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T13:21:28.347 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.347 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.347 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-03-23T13:21:28.348 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-03-23T13:21:28.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.348 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.348 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-03-23T13:21:28.348 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.349 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T13:21:28.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.349 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-03-23T13:21:28.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-03-23T13:21:28.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.350 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.350 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.350 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.350 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T13:21:28.350 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T13:21:28.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.351 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.351 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-03-23T13:21:28.351 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-03-23T13:21:28.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.351 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:28.352 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.352 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.352 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.352 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T13:21:28.352 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.352 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.353 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-03-23T13:21:28.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T13:21:28.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:28.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.353 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-03-23T13:21:28.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T13:21:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-03-23T13:21:28.354 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-03-23T13:21:28.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.355 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.355 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.356 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-03-23T13:21:28.356 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-03-23T13:21:28.356 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-03-23T13:21:28.356 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.356 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-23T13:21:28.356 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.357 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.357 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-03-23T13:21:28.357 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-03-23T13:21:28.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.357 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.358 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.358 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.358 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.358 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T13:21:28.358 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.359 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-03-23T13:21:28.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-03-23T13:21:28.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.359 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.359 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T13:21:28.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.360 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.360 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-03-23T13:21:28.360 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-03-23T13:21:28.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.361 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.361 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.361 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.361 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.361 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T13:21:28.361 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.362 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.362 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-03-23T13:21:28.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-03-23T13:21:28.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.362 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.362 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T13:21:28.363 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.363 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.363 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.363 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:28.363 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.363 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.363 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-03-23T13:21:28.364 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-03-23T13:21:28.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.364 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.364 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T13:21:28.364 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.364 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.365 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-03-23T13:21:28.365 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.365 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.365 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-03-23T13:21:28.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-03-23T13:21:28.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.366 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T13:21:28.366 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.366 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-03-23T13:21:28.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.367 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-03-23T13:21:28.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-03-23T13:21:28.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.367 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.367 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-23T13:21:28.367 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.368 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:28.368 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.368 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.368 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-03-23T13:21:28.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-03-23T13:21:28.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.369 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.369 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T13:21:28.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.370 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.370 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-03-23T13:21:28.370 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.370 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.370 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.370 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-03-23T13:21:28.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-03-23T13:21:28.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.371 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.371 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T13:21:28.371 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.371 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.371 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.372 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-03-23T13:21:28.372 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.372 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.372 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-03-23T13:21:28.372 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-03-23T13:21:28.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.373 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.373 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T13:21:28.373 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.373 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.373 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-03-23T13:21:28.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.374 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.374 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-03-23T13:21:28.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-03-23T13:21:28.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.374 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.374 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.374 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T13:21:28.375 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.375 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.375 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-03-23T13:21:28.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.375 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-03-23T13:21:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-03-23T13:21:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T13:21:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-03-23T13:21:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-03-23T13:21:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-03-23T13:21:28.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.378 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T13:21:28.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.378 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.379 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-03-23T13:21:28.379 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.379 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.379 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.379 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-03-23T13:21:28.379 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-03-23T13:21:28.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.379 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.380 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T13:21:28.380 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.380 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.380 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:28.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.381 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.381 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-03-23T13:21:28.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-03-23T13:21:28.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.381 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-23T13:21:28.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.382 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.382 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-03-23T13:21:28.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.382 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.382 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-03-23T13:21:28.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-03-23T13:21:28.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T13:21:28.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.383 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.383 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:28.384 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.384 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-03-23T13:21:28.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-03-23T13:21:28.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.384 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T13:21:28.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.385 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-03-23T13:21:28.385 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.386 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-03-23T13:21:28.386 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-03-23T13:21:28.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.386 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.386 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T13:21:28.386 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-03-23T13:21:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-03-23T13:21:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-03-23T13:21:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T13:21:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-03-23T13:21:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-03-23T13:21:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:28.390 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.390 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.390 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T13:21:28.390 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.390 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.391 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-03-23T13:21:28.391 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T13:21:28.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.391 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.391 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-03-23T13:21:28.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.391 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.392 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T13:21:28.392 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.392 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-03-23T13:21:28.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-03-23T13:21:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-03-23T13:21:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T13:21:28.393 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-03-23T13:21:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-03-23T13:21:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.395 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-03-23T13:21:28.395 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.395 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-03-23T13:21:28.397 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.398 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.398 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-03-23T13:21:28.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-03-23T13:21:28.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.398 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.399 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.399 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.399 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.399 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T13:21:28.399 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.399 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.400 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-03-23T13:21:28.400 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-03-23T13:21:28.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.400 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.400 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:28.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.400 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.400 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.401 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-03-23T13:21:28.401 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.401 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-03-23T13:21:28.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-03-23T13:21:28.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.401 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-03-23T13:21:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-03-23T13:21:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-03-23T13:21:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T13:21:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-03-23T13:21:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-03-23T13:21:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T13:21:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-03-23T13:21:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-03-23T13:21:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T13:21:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T13:21:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.412 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.412 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.413 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.413 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.413 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.413 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.413 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.414 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.414 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.414 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.414 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.414 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.414 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.415 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.415 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.415 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.416 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.416 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.416 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.416 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.416 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.416 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T13:21:28.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T13:21:28.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.417 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.417 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.417 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.417 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.417 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T13:21:28.417 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.418 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.418 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T13:21:28.418 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T13:21:28.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.418 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-03-23T13:21:28.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.419 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.419 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T13:21:28.419 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.419 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-03-23T13:21:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-03-23T13:21:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.421 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-03-23T13:21:28.421 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.421 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.421 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-03-23T13:21:28.421 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-03-23T13:21:28.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.422 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:28.422 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.422 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.422 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T13:21:28.422 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.423 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.423 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-03-23T13:21:28.423 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-03-23T13:21:28.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.423 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.423 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.423 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.424 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.424 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.424 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-03-23T13:21:28.424 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.424 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.424 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-23T13:21:28.424 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-23T13:21:28.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.425 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-03-23T13:21:28.425 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.425 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.425 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.425 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-03-23T13:21:28.426 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-03-23T13:21:28.426 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.426 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T13:21:28.426 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.426 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-03-23T13:21:28.426 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-03-23T13:21:28.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.427 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.427 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-03-23T13:21:28.427 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.427 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.427 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T13:21:28.427 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.428 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.428 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-03-23T13:21:28.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-03-23T13:21:28.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.428 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.429 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T13:21:28.429 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.429 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.429 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.429 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T13:21:28.429 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.429 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.430 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-03-23T13:21:28.430 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-03-23T13:21:28.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.430 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.431 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.431 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T13:21:28.431 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.431 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-03-23T13:21:28.432 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-03-23T13:21:28.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.432 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.432 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.432 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.432 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.433 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T13:21:28.433 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.433 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.433 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-03-23T13:21:28.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-03-23T13:21:28.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.434 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.434 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.434 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.434 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.434 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.434 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T13:21:28.435 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.435 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.435 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-03-23T13:21:28.435 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-03-23T13:21:28.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.435 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.436 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.436 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.436 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.436 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.436 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T13:21:28.436 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.437 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.437 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-03-23T13:21:28.437 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-03-23T13:21:28.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.437 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.437 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.438 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.438 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.438 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.438 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T13:21:28.438 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.438 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.439 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-03-23T13:21:28.439 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-03-23T13:21:28.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.439 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.439 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-03-23T13:21:28.439 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.439 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.439 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.440 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-03-23T13:21:28.440 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.440 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.440 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-03-23T13:21:28.440 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-03-23T13:21:28.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.441 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.441 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.441 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.441 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.441 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.441 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T13:21:28.441 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.442 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.442 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-03-23T13:21:28.442 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-03-23T13:21:28.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.442 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.443 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.443 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.443 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.443 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T13:21:28.443 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.443 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.444 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-03-23T13:21:28.444 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-03-23T13:21:28.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.444 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.444 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.444 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.444 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.444 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.445 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T13:21:28.445 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.445 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.445 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-03-23T13:21:28.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-03-23T13:21:28.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.446 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.446 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.446 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T13:21:28.446 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.447 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.447 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-03-23T13:21:28.447 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-03-23T13:21:28.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.447 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T13:21:28.448 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.448 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.448 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-03-23T13:21:28.448 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-03-23T13:21:28.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.449 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.449 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.449 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.449 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.449 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.449 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T13:21:28.449 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.450 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.450 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-03-23T13:21:28.450 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.451 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.451 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.451 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.451 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.451 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.451 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.451 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.452 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.452 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.452 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.452 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.452 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.452 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.452 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.452 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.452 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.453 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.453 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.453 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.453 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.453 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.454 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.454 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.454 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.454 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.455 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.455 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.455 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.456 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.456 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.456 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.456 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.456 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.456 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.456 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.456 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.456 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.457 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.457 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.457 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.457 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.457 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.457 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.458 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.458 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.458 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.458 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.458 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.458 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.458 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.459 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-03-23T13:21:28.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-03-23T13:21:28.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.459 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.459 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-03-23T13:21:28.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.460 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.460 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T13:21:28.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.460 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.460 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-03-23T13:21:28.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-03-23T13:21:28.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-03-23T13:21:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T13:21:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-03-23T13:21:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.463 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T13:21:28.463 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T13:21:28.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.463 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:28.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.463 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.464 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.464 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.464 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.464 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.464 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T13:21:28.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T13:21:28.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.465 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.465 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.465 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.466 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T13:21:28.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T13:21:28.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.466 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.466 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.467 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.467 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.467 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.467 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.467 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.467 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.467 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-03-23T13:21:28.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T13:21:28.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.468 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.468 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.468 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.468 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.468 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.469 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.469 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T13:21:28.469 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T13:21:28.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.469 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.470 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.470 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.470 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.470 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.470 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.470 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.471 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T13:21:28.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T13:21:28.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.471 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.471 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.471 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.472 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.472 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.472 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.472 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.472 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.472 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T13:21:28.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T13:21:28.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.473 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.473 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.473 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.473 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.473 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.474 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.474 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T13:21:28.474 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T13:21:28.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.474 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.475 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.475 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.475 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.475 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.475 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T13:21:28.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T13:21:28.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.476 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.476 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.476 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.476 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.477 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.477 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.477 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.477 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.478 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T13:21:28.478 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T13:21:28.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.478 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.478 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.480 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.480 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T13:21:28.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T13:21:28.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.481 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.481 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.481 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.481 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.482 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.482 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.482 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.482 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T13:21:28.482 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T13:21:28.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T13:21:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T13:21:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T13:21:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T13:21:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T13:21:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T13:21:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-23T13:21:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T13:21:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T13:21:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-03-23T13:21:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-03-23T13:21:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T13:21:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T13:21:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-23T13:21:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T13:21:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T13:21:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T13:21:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T13:21:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T13:21:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T13:21:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T13:21:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T13:21:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T13:21:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T13:21:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T13:21:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T13:21:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.503 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.503 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.503 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.503 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T13:21:28.503 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T13:21:28.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.503 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.504 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.504 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.504 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.504 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.504 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.504 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.504 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.505 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T13:21:28.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T13:21:28.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.505 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.505 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.505 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.505 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.505 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.506 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.506 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.506 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.506 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T13:21:28.506 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T13:21:28.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.506 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.507 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.507 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.507 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.507 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.507 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.507 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T13:21:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T13:21:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T13:21:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T13:21:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T13:21:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T13:21:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T13:21:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T13:21:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T13:21:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T13:21:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T13:21:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T13:21:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T13:21:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T13:21:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T13:21:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T13:21:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T13:21:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-03-23T13:21:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T13:21:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T13:21:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-03-23T13:21:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T13:21:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T13:21:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T13:21:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T13:21:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-23T13:21:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T13:21:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T13:21:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T13:21:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T13:21:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T13:21:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T13:21:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T13:21:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T13:21:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T13:21:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T13:21:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-03-23T13:21:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-03-23T13:21:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-03-23T13:21:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-03-23T13:21:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-03-23T13:21:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-03-23T13:21:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-03-23T13:21:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-03-23T13:21:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-03-23T13:21:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-03-23T13:21:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-23T13:21:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-23T13:21:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-03-23T13:21:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-03-23T13:21:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-03-23T13:21:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-03-23T13:21:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T13:21:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-23T13:21:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-23T13:21:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-23T13:21:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-23T13:21:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-23T13:21:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-03-23T13:21:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-23T13:21:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-23T13:21:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-03-23T13:21:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-03-23T13:21:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-03-23T13:21:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-23T13:21:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-23T13:21:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-03-23T13:21:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-03-23T13:21:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-23T13:21:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-23T13:21:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-23T13:21:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-23T13:21:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-23T13:21:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-03-23T13:21:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-03-23T13:21:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-23T13:21:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-03-23T13:21:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-03-23T13:21:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-03-23T13:21:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-03-23T13:21:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-03-23T13:21:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-03-23T13:21:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T13:21:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-03-23T13:21:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-03-23T13:21:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-03-23T13:21:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T13:21:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-03-23T13:21:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-03-23T13:21:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-03-23T13:21:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-03-23T13:21:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-03-23T13:21:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-03-23T13:21:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-23T13:21:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T13:21:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-03-23T13:21:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-03-23T13:21:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.583 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.583 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.583 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T13:21:28.583 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.584 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:28.584 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:28.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.584 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.584 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:28.584 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.585 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.585 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T13:21:28.585 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.585 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-03-23T13:21:28.586 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-03-23T13:21:28.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.586 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.586 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-03-23T13:21:28.586 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.586 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.587 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T13:21:28.587 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.587 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.587 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-03-23T13:21:28.588 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.588 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.588 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.588 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.588 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.588 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.588 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.588 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.589 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.589 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.589 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.589 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.589 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.589 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.589 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.589 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.589 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.590 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.590 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.590 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.590 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.591 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.591 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.591 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.591 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.591 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.591 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.592 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.592 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.592 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.592 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.592 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.592 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.592 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.593 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.593 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.593 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.593 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.593 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.593 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.593 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.593 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.594 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.594 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.594 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.594 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.594 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.595 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.595 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.595 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.595 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.595 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.595 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.595 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.596 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-03-23T13:21:28.596 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-03-23T13:21:28.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.596 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.596 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-23T13:21:28.596 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.596 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.597 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.597 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.597 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.597 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-03-23T13:21:28.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-03-23T13:21:28.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.597 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.598 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-03-23T13:21:28.598 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.598 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.598 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.598 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.599 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:28.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:28.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:28.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.600 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.600 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.600 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.600 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.600 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-03-23T13:21:28.601 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-03-23T13:21:28.601 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-03-23T13:21:28.601 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.601 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-03-23T13:21:28.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-03-23T13:21:28.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.601 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.602 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-03-23T13:21:28.602 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.602 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.602 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.602 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T13:21:28.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.602 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-03-23T13:21:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-03-23T13:21:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-03-23T13:21:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.604 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T13:21:28.604 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.604 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.604 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-03-23T13:21:28.604 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-03-23T13:21:28.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.604 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.605 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.605 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.605 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-03-23T13:21:28.605 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.605 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.606 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:28.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:28.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.606 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.606 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-03-23T13:21:28.606 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-03-23T13:21:28.606 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.607 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.607 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T13:21:28.607 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.607 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.607 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.607 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.608 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.608 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.608 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.608 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.608 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.608 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.609 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.609 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.609 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.609 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.609 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.609 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.610 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.610 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.610 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.610 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.611 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.611 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.611 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.611 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.611 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.612 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.612 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.612 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.612 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.612 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.612 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.612 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.613 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.614 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.614 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.614 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.614 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.614 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.615 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.615 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.615 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.615 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.616 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.616 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-03-23T13:21:28.616 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-03-23T13:21:28.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.616 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.616 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.616 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.617 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.617 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.617 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-03-23T13:21:28.617 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.617 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.617 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-03-23T13:21:28.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-03-23T13:21:28.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.618 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-03-23T13:21:28.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.618 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.618 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-03-23T13:21:28.618 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.619 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.619 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.619 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-23T13:21:28.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-23T13:21:28.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.619 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.619 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-03-23T13:21:28.619 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.620 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.620 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.620 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T13:21:28.620 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.620 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.620 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-03-23T13:21:28.620 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-03-23T13:21:28.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.621 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.621 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:28.621 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:28.621 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-03-23T13:21:28.621 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.621 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T13:21:28.621 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.622 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.622 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-03-23T13:21:28.622 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-03-23T13:21:28.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.622 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.622 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.622 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.623 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.623 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.623 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-03-23T13:21:28.623 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.623 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.623 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-03-23T13:21:28.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-03-23T13:21:28.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.624 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.624 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-03-23T13:21:28.624 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.624 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.624 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T13:21:28.625 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.625 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.625 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-03-23T13:21:28.625 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.625 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.626 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.626 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.626 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.626 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.626 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.626 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.626 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.627 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.627 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.627 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.627 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.627 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.627 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.627 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.627 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.628 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.628 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.628 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.628 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.628 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.628 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.629 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.629 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.629 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.629 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.629 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.629 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.630 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.630 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.630 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.630 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.630 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.631 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.631 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.631 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.631 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.631 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.631 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.631 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.631 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.631 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.632 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.632 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.632 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.632 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.632 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.632 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.633 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.633 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.633 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.633 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.633 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.633 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-03-23T13:21:28.634 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.634 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.634 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.634 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.635 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.635 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.635 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.635 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.635 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.635 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.636 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.636 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.636 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.636 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.636 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.636 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.636 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.636 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.637 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.637 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.637 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.637 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.638 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.638 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.638 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.638 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.638 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.638 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.639 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.639 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.639 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.639 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.639 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.639 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.639 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.640 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.640 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.640 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.640 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.640 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.640 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.640 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.640 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-03-23T13:21:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-03-23T13:21:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-03-23T13:21:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-03-23T13:21:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T13:21:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-03-23T13:21:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-03-23T13:21:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-03-23T13:21:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T13:21:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-03-23T13:21:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-03-23T13:21:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-03-23T13:21:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-03-23T13:21:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-03-23T13:21:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T13:21:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.663 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.663 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.663 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.663 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.663 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.663 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.663 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.664 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.664 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.664 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.664 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.664 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.665 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.665 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.665 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.665 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.665 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.665 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.665 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.666 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.666 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.666 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.666 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.666 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-03-23T13:21:28.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-03-23T13:21:28.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.667 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.667 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.667 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.667 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.667 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-03-23T13:21:28.667 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T13:21:28.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.668 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.668 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-03-23T13:21:28.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-03-23T13:21:28.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.668 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.668 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-03-23T13:21:28.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.669 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.669 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-03-23T13:21:28.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.669 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.670 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-03-23T13:21:28.670 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-03-23T13:21:28.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.670 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-03-23T13:21:28.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.670 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.670 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.671 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T13:21:28.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.671 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-03-23T13:21:28.671 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-03-23T13:21:28.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.672 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.672 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.672 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.672 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.672 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.672 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-03-23T13:21:28.672 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.673 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.673 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-03-23T13:21:28.673 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-03-23T13:21:28.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.673 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.673 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-03-23T13:21:28.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.674 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.674 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.674 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T13:21:28.674 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T13:21:28.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.674 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-03-23T13:21:28.675 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.675 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.675 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.675 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.676 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.676 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.676 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.676 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.676 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.677 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.677 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.677 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.677 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.677 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.678 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.678 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.678 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.678 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.679 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.679 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.679 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.679 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.679 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.679 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.680 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.681 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.681 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.681 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.681 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.681 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.681 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.681 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.681 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.682 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.682 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.682 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.682 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.682 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.683 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.683 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.683 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.683 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.683 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.684 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-03-23T13:21:28.684 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.685 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.685 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.685 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.686 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.686 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.686 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.686 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.686 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.686 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.686 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.687 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.687 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.687 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.688 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.688 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.688 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.688 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.688 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.688 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.688 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.689 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.689 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.689 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.689 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.689 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.690 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.690 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.690 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.690 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.690 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.690 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.690 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.690 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.690 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.691 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.691 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.691 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.691 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.691 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.692 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.692 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.692 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.692 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.692 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.692 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.693 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.693 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-03-23T13:21:28.693 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.694 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.694 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.694 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.694 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.694 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.695 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.695 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.695 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.695 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.695 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.695 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.695 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.695 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.695 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.696 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.696 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.696 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.696 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.696 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.696 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.697 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.697 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.697 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.697 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.697 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.697 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.697 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.698 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.698 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.698 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.698 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.699 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.699 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.699 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.699 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.699 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.699 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.699 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.699 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.699 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.700 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.700 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.700 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.700 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.700 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.700 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.701 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.701 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.701 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.701 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.701 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.701 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.701 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.701 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.702 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-23T13:21:28.702 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.702 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.702 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.702 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-03-23T13:21:28.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-03-23T13:21:28.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.703 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.703 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.703 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.703 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.703 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.703 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T13:21:28.704 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.704 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.704 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.704 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.705 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.705 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.705 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.705 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.705 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.705 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.705 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.705 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.706 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.706 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.706 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.706 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.707 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.707 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.707 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.707 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.708 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.708 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.708 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.708 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.708 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.709 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.709 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.709 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.709 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.709 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.709 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.710 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.710 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.710 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.710 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.710 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.710 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.710 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.710 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.711 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.711 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.711 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.711 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.712 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.712 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.712 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.712 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.712 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-03-23T13:21:28.712 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-23T13:21:28.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.713 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.713 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.713 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.713 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.713 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.713 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.713 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-23T13:21:28.713 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-03-23T13:21:28.713 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-03-23T13:21:28.714 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.714 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-03-23T13:21:28.714 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.714 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.714 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-03-23T13:21:28.715 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.715 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.715 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.715 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.716 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.716 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.716 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.716 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.716 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.716 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.716 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.717 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.717 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.717 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.717 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.717 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.717 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.717 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.717 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.718 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.718 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.718 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.718 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.719 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.719 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.719 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.719 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.719 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.719 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.719 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.720 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.721 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.722 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.722 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.722 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.722 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.722 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.723 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.723 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.723 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.724 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-03-23T13:21:28.724 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.724 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.724 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.724 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.724 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.725 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.725 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.725 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.725 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.725 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.725 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.725 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.725 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.725 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.726 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.726 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.726 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.726 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.726 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.726 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.727 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.727 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.727 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.727 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.727 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.727 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.728 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.728 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.728 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.728 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.728 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.729 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.729 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.729 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.729 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.729 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.729 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.729 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.729 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.729 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.730 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.730 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.730 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.730 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.730 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.730 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.730 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.731 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.731 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.731 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.732 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.732 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.732 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-03-23T13:21:28.733 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.733 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.733 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.733 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.734 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.734 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.734 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.734 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.734 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.735 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.735 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.735 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.735 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.735 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.735 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.735 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.736 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.736 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.736 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.736 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.736 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.737 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.737 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.737 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.737 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.737 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.737 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.738 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.738 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.738 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.738 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.738 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.738 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.738 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.739 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.739 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.739 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.739 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.739 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.739 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.739 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.740 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.740 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.740 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.740 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.741 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.741 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.741 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.741 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:28.741 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:28.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.741 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.742 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:28.742 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.742 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.742 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.742 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.742 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.743 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.743 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-03-23T13:21:28.743 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-03-23T13:21:28.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:28.743 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.743 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-23T13:21:28.744 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-03-23T13:21:28.744 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.744 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.744 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-03-23T13:21:28.744 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T13:21:28.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.744 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.745 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-03-23T13:21:28.745 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.745 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.745 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.745 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-03-23T13:21:28.745 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-03-23T13:21:28.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.746 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.746 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.746 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.746 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.746 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.746 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T13:21:28.747 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.747 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.747 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.747 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-23T13:21:28.747 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-23T13:21:28.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.747 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.748 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.748 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.748 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.748 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.748 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.748 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.748 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.748 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.749 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.749 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.749 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.749 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.750 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.750 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.750 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.750 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.750 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.750 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.750 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.751 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.751 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.751 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.752 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.752 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.752 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.752 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.752 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.752 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.753 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.753 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.753 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.753 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.754 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.754 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.754 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.755 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.755 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.755 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.755 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.755 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.755 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.755 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.756 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.756 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.756 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.756 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.756 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.756 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.757 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.757 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.757 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.757 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.757 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.757 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.757 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.758 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.758 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-03-23T13:21:28.758 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-03-23T13:21:28.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.758 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.759 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-03-23T13:21:28.759 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.759 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.759 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.759 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-03-23T13:21:28.759 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.760 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:28.760 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:28.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.760 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.760 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:28.760 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.761 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.761 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.761 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-03-23T13:21:28.761 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.761 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.762 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-03-23T13:21:28.762 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.762 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.762 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.762 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-03-23T13:21:28.762 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-03-23T13:21:28.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.763 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.763 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.763 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.763 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.763 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.763 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.763 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.764 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.764 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-03-23T13:21:28.764 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-03-23T13:21:28.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.764 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.765 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.765 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.765 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.765 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.765 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T13:21:28.765 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T13:21:28.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.765 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.766 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.766 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.766 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.766 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.766 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.768 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.768 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.768 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.768 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.768 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.769 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.769 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.769 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.769 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.769 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.769 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.769 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.770 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.770 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.770 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.770 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.770 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.770 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.771 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.771 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.771 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.771 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.771 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.771 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.771 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.771 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.772 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.772 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.772 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.772 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.772 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.772 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.773 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.773 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.773 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.773 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.773 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.773 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.773 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-03-23T13:21:28.774 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-03-23T13:21:28.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.774 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.774 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.774 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.775 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.775 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.775 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.775 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.775 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T13:21:28.776 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T13:21:28.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.776 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.776 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-03-23T13:21:28.776 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.776 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.776 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.776 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.777 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.777 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.777 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-03-23T13:21:28.777 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-03-23T13:21:28.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:28.777 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.778 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-03-23T13:21:28.778 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.778 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.778 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.778 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.778 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.779 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.779 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-23T13:21:28.779 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-23T13:21:28.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.779 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.779 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.779 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-23T13:21:28.779 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.780 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.780 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.780 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T13:21:28.780 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.780 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.780 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.780 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-23T13:21:28.781 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-23T13:21:28.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.781 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.781 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-03-23T13:21:28.781 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.781 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.781 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.781 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T13:21:28.782 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.782 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.782 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-03-23T13:21:28.782 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-03-23T13:21:28.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.782 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.783 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-03-23T13:21:28.783 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.783 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.783 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.783 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.783 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.784 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.784 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-23T13:21:28.784 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-23T13:21:28.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.784 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.785 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-03-23T13:21:28.785 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.785 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.785 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.785 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-03-23T13:21:28.785 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-03-23T13:21:28.785 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.785 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.786 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.786 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.786 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-23T13:21:28.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-23T13:21:28.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.787 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.787 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-23T13:21:28.787 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-23T13:21:28.787 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-23T13:21:28.787 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.787 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.788 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.788 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-23T13:21:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.790 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.790 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.790 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.790 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.790 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.790 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.791 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.791 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.791 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.791 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.792 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.792 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.792 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.793 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.793 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.793 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.794 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.794 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.795 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.795 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.795 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.796 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.796 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.796 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.796 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.797 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.797 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.798 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.798 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.798 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.798 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.798 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.799 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.799 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.799 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.799 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.799 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.800 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.800 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.800 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.800 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.800 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.801 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.801 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.801 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.801 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.801 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.801 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.801 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.802 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.802 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-03-23T13:21:28.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-03-23T13:21:28.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.802 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.802 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:28.803 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.803 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.803 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.803 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.803 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.803 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.804 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-03-23T13:21:28.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T13:21:28.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.804 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.804 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.804 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.804 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.804 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.805 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.805 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.805 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.805 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-03-23T13:21:28.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-03-23T13:21:28.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.805 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.806 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.806 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.806 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.806 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.809 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.809 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.809 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.810 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-03-23T13:21:28.810 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.810 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.811 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.811 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-03-23T13:21:28.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-03-23T13:21:28.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.811 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.811 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.811 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.811 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.812 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.812 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.812 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.812 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.812 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-23T13:21:28.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-23T13:21:28.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.813 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.813 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.813 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.813 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.813 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.813 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.813 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.814 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.814 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.814 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.814 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.815 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.815 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.815 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.815 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.815 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.815 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.815 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.815 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.815 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.816 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.816 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.816 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.816 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.817 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.817 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.817 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.818 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.818 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.818 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.818 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.818 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.819 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.819 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.819 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.819 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.819 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.819 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.819 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.820 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.820 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.820 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.820 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.820 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.820 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.820 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.821 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.821 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.821 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.821 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.821 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.821 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.822 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.822 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.822 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-23T13:21:28.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-23T13:21:28.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.822 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.823 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.823 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.823 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.823 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.823 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.823 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.823 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.824 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-23T13:21:28.824 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-23T13:21:28.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.824 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.824 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.824 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.825 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.825 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.825 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.825 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.825 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.826 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-03-23T13:21:28.826 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.826 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.826 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.826 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.826 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.826 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.827 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.827 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.827 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.827 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.827 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.828 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.828 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.828 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.828 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.828 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.828 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.828 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.829 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.829 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.829 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.829 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.829 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.829 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.830 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.830 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.830 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.830 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.831 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.831 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.831 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.831 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.831 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.831 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.831 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.831 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.832 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.832 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.832 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.832 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.832 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.832 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.832 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.833 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.833 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.833 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.833 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.833 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.833 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.834 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.834 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.834 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.834 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-03-23T13:21:28.835 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.835 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.835 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.835 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.836 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.836 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.836 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.836 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.836 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.837 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.837 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.837 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.837 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.837 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.837 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.837 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.838 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.838 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.838 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.838 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.839 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.839 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.839 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.839 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.839 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.839 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.839 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.840 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.840 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.840 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.840 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.841 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.841 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.841 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.841 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.841 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.841 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.842 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.842 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.842 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.842 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.842 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.842 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.842 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.843 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.843 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.843 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.843 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.844 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.844 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.844 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.844 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.844 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.845 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.845 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-03-23T13:21:28.845 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-03-23T13:21:28.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.845 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-03-23T13:21:28.846 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.846 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.846 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.846 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T13:21:28.846 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.846 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-03-23T13:21:28.847 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.847 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.847 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.847 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.848 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.848 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.848 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.848 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.848 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.848 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.849 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.849 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.849 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.849 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.849 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.849 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.849 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.849 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.850 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.850 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.850 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.850 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.851 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.851 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.851 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.851 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.851 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.851 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.851 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.852 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.852 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.852 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.852 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.853 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.853 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.853 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.853 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.853 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.853 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.853 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.853 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.854 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.854 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.854 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.854 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.854 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.854 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.855 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.855 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.855 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.855 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.855 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.855 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.856 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.856 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.856 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.856 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-03-23T13:21:28.856 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-03-23T13:21:28.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.856 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.857 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.857 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.857 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.857 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.857 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.857 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.858 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.858 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-03-23T13:21:28.858 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-03-23T13:21:28.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.858 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.858 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-03-23T13:21:28.859 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.859 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.859 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.859 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.859 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.859 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.859 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-03-23T13:21:28.860 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-03-23T13:21:28.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.860 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.860 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.860 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.860 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.860 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.860 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.861 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.861 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.861 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-03-23T13:21:28.861 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-03-23T13:21:28.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.861 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.861 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.862 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.862 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.862 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.862 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.862 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.862 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.862 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.863 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-03-23T13:21:28.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-03-23T13:21:28.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:28.863 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.863 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.863 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.863 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.864 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.864 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.864 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.864 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.864 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-03-23T13:21:28.865 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-03-23T13:21:28.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.865 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.865 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.865 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.865 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.865 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.865 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.865 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.866 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.866 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.866 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-03-23T13:21:28.866 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-03-23T13:21:28.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.866 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.867 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.867 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.867 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.867 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.867 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.867 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.867 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.868 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-03-23T13:21:28.868 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-03-23T13:21:28.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.868 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.869 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.869 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.869 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.869 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.869 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-03-23T13:21:28.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-03-23T13:21:28.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.870 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.870 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.870 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:28.870 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-03-23T13:21:28.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.870 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.871 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.871 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.871 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-03-23T13:21:28.871 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-03-23T13:21:28.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.871 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.872 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.872 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.872 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.872 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.872 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.872 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.873 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-03-23T13:21:28.873 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-03-23T13:21:28.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.873 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.873 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.873 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.873 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.873 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.874 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.874 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.874 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.874 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-03-23T13:21:28.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-03-23T13:21:28.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.874 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.875 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.875 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.875 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.875 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.875 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.875 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.875 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.876 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-03-23T13:21:28.876 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.876 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.876 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.876 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-03-23T13:21:28.877 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-03-23T13:21:28.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.877 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.877 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.877 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.877 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.877 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.877 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T13:21:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-03-23T13:21:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-03-23T13:21:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.879 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.879 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.879 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.879 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.879 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-03-23T13:21:28.879 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.879 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.880 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.880 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.880 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.880 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.880 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.880 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.881 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.881 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.881 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.881 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.881 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.881 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.881 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.882 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.882 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.882 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.882 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.882 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.883 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.883 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.883 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.883 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.883 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.883 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.884 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.884 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.884 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.884 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.884 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.885 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.885 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.885 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.885 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.885 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.885 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.885 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.885 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.886 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.886 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.886 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.886 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.886 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.886 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.886 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.887 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.887 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.887 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.887 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.887 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.888 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.888 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.888 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.888 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-03-23T13:21:28.889 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.889 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.889 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.889 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.890 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.890 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.890 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.890 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.890 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.891 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.891 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.891 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.891 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.891 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.891 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.891 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.891 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.891 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.892 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.892 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.892 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.892 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.893 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.893 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.893 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.893 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.893 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.893 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.893 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.894 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.894 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.894 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.894 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.895 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.895 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.895 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.895 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.895 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.895 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.895 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.895 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.896 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.896 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.896 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.896 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.896 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.896 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.897 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.897 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.897 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.897 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.897 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.897 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.897 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.898 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-03-23T13:21:28.898 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.899 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.899 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.899 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.899 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.899 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.900 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.900 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.900 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.900 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.900 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.900 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.900 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.900 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.900 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.901 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.901 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.901 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.901 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.901 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.902 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.902 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.902 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.902 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.902 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.902 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.902 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.903 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.903 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.904 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.904 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.904 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.904 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.904 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.905 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.905 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.905 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.905 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.905 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.906 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.906 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.906 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.906 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.907 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.907 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.907 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.907 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.908 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-03-23T13:21:28.908 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.908 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.908 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.908 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-03-23T13:21:28.908 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-03-23T13:21:28.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.908 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T13:21:28.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.909 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.909 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.909 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.909 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.909 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.910 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:28.910 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:28.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.910 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.910 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-03-23T13:21:28.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.911 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.911 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.911 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.911 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.912 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.912 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.912 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.912 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.912 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.912 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.912 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.913 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.913 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.913 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.913 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.913 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.913 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.913 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.913 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.914 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.914 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.915 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.915 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.915 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.915 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.915 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.915 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.916 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.916 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.916 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.916 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.916 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.917 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.917 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.917 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.917 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.917 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.917 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.917 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.917 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.917 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.918 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.918 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.918 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.918 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.919 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.919 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.919 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.919 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-03-23T13:21:28.920 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.920 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.920 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.920 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-03-23T13:21:28.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-03-23T13:21:28.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.921 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.921 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.922 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T13:21:28.922 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.922 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.922 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-03-23T13:21:28.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-03-23T13:21:28.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.923 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.923 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.923 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.923 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T13:21:28.923 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.924 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.924 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-03-23T13:21:28.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-03-23T13:21:28.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.924 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.925 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.925 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T13:21:28.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.926 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-03-23T13:21:28.926 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-03-23T13:21:28.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.926 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.927 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.927 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T13:21:28.927 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.927 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.927 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-03-23T13:21:28.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-03-23T13:21:28.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.928 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.928 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.928 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.928 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.928 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T13:21:28.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.929 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-03-23T13:21:28.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-03-23T13:21:28.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.929 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.930 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.930 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.930 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.930 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.931 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-03-23T13:21:28.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-03-23T13:21:28.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.931 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.931 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.931 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.931 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.931 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.932 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.932 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-03-23T13:21:28.932 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-03-23T13:21:28.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.932 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T13:21:28.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.933 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.933 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.933 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.933 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.934 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-23T13:21:28.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-23T13:21:28.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.934 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.934 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.935 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.935 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.935 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.935 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.935 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.935 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.936 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:28.936 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:28.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.936 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.936 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-03-23T13:21:28.936 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-03-23T13:21:28.936 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.936 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.936 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.937 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.937 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-03-23T13:21:28.937 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-03-23T13:21:28.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.938 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-03-23T13:21:28.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.938 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.938 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.938 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.939 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.939 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-03-23T13:21:28.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-03-23T13:21:28.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.939 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.939 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.940 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.940 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.940 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.940 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.940 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.940 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-03-23T13:21:28.940 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-03-23T13:21:28.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.941 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.941 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.942 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.942 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.942 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.942 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.942 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-03-23T13:21:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-03-23T13:21:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-03-23T13:21:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-03-23T13:21:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-03-23T13:21:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-03-23T13:21:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T13:21:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-03-23T13:21:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-23T13:21:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-03-23T13:21:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-03-23T13:21:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-03-23T13:21:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T13:21:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-03-23T13:21:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-03-23T13:21:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-03-23T13:21:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-03-23T13:21:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-03-23T13:21:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T13:21:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-03-23T13:21:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T13:21:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T13:21:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-03-23T13:21:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-03-23T13:21:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-03-23T13:21:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-03-23T13:21:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-03-23T13:21:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-03-23T13:21:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-03-23T13:21:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-03-23T13:21:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-03-23T13:21:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-03-23T13:21:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-03-23T13:21:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-03-23T13:21:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-03-23T13:21:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T13:21:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-23T13:21:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24116, 2024-03-23T13:21:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "y", 2024-03-23T13:21:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-03-23T13:21:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-03-23T13:21:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-03-23T13:21:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T13:21:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-03-23T13:21:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-03-23T13:21:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.015 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.015 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-03-23T13:21:29.015 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.016 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-03-23T13:21:29.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-03-23T13:21:29.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.016 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-03-23T13:21:29.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-03-23T13:21:29.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.017 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.018 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-03-23T13:21:29.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-03-23T13:21:29.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.018 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.018 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.019 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-03-23T13:21:29.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.019 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.019 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-03-23T13:21:29.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-03-23T13:21:29.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.020 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.020 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.020 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-03-23T13:21:29.021 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.021 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.021 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-03-23T13:21:29.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-03-23T13:21:29.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.021 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.022 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-03-23T13:21:29.022 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.022 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.022 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-03-23T13:21:29.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.023 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.023 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-03-23T13:21:29.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-03-23T13:21:29.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.024 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.024 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.024 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-03-23T13:21:29.024 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.025 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.025 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-03-23T13:21:29.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-03-23T13:21:29.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.025 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.025 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.026 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.026 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.026 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T13:21:29.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.027 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-03-23T13:21:29.027 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-03-23T13:21:29.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-03-23T13:21:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-03-23T13:21:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-03-23T13:21:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-03-23T13:21:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T13:21:29.030 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.031 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-03-23T13:21:29.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-03-23T13:21:29.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-03-23T13:21:29.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.031 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T13:21:29.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T13:21:29.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.032 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.032 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-03-23T13:21:29.032 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-03-23T13:21:29.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:29.032 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:29.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-23T13:21:29.033 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-23T13:21:29.033 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.033 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T13:21:29.033 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T13:21:29.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.034 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.034 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-03-23T13:21:29.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-03-23T13:21:29.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:29.034 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.034 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-03-23T13:21:29.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:29.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-03-23T13:21:29.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T13:21:29.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.035 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.036 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-03-23T13:21:29.036 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-03-23T13:21:29.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.036 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T13:21:29.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.037 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.037 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.037 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T13:21:29.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T13:21:29.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.037 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-03-23T13:21:29.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-03-23T13:21:29.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-03-23T13:21:29.038 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-03-23T13:21:29.038 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-03-23T13:21:29.038 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.038 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-03-23T13:21:29.038 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T13:21:29.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.039 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-03-23T13:21:29.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-03-23T13:21:29.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.039 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-03-23T13:21:29.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.040 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.040 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.040 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T13:21:29.040 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T13:21:29.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.040 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.040 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-03-23T13:21:29.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-03-23T13:21:29.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:29.041 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.041 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:29.041 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-23T13:21:29.041 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-23T13:21:29.041 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T13:21:29.042 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T13:21:29.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.042 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.042 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.042 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.043 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.043 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.043 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.043 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.043 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.043 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.043 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.043 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.044 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.044 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.044 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.045 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.046 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.047 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.047 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.048 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.048 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.048 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.048 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.048 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.048 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.048 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.049 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.049 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.050 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.050 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.050 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-03-23T13:21:29.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-03-23T13:21:29.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:29.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T13:21:29.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.053 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-23T13:21:29.053 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-23T13:21:29.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.053 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-03-23T13:21:29.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.054 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.054 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-03-23T13:21:29.054 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-03-23T13:21:29.054 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-03-23T13:21:29.054 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-03-23T13:21:29.054 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-03-23T13:21:29.054 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-03-23T13:21:29.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.055 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.055 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-03-23T13:21:29.055 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-03-23T13:21:29.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.055 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.056 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.056 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.056 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.056 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T13:21:29.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.057 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.057 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:29.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:29.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.057 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.058 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:29.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.058 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.058 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.058 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T13:21:29.058 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.059 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-03-23T13:21:29.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-03-23T13:21:29.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:29.060 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:29.060 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T13:21:29.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T13:21:29.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.061 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-03-23T13:21:29.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-03-23T13:21:29.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:29.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.062 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T13:21:29.062 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-03-23T13:21:29.063 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.063 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.063 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.063 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-03-23T13:21:29.063 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-03-23T13:21:29.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.063 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-23T13:21:29.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.064 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T13:21:29.064 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.064 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-03-23T13:21:29.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-03-23T13:21:29.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.065 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-03-23T13:21:29.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T13:21:29.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.066 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-03-23T13:21:29.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-03-23T13:21:29.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.067 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-03-23T13:21:29.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T13:21:29.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.068 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-03-23T13:21:29.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-03-23T13:21:29.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.069 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T13:21:29.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T13:21:29.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.069 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-03-23T13:21:29.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-03-23T13:21:29.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.070 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:29.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T13:21:29.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.071 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-03-23T13:21:29.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T13:21:29.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.071 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-03-23T13:21:29.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T13:21:29.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.072 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.073 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-03-23T13:21:29.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-03-23T13:21:29.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.073 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.073 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.074 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-03-23T13:21:29.074 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-03-23T13:21:29.074 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-03-23T13:21:29.074 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-23T13:21:29.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.075 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-03-23T13:21:29.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-03-23T13:21:29.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T13:21:29.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.077 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-03-23T13:21:29.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-03-23T13:21:29.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.078 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T13:21:29.078 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.078 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.079 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-03-23T13:21:29.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-03-23T13:21:29.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T13:21:29.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.080 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-03-23T13:21:29.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-03-23T13:21:29.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T13:21:29.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:29.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.081 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.082 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-03-23T13:21:29.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-03-23T13:21:29.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T13:21:29.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-03-23T13:21:29.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.083 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.083 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-03-23T13:21:29.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-03-23T13:21:29.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.084 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.084 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T13:21:29.084 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.084 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.084 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-03-23T13:21:29.085 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.085 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.085 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-03-23T13:21:29.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-03-23T13:21:29.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.086 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-23T13:21:29.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:29.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.086 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.087 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-03-23T13:21:29.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-03-23T13:21:29.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.087 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T13:21:29.087 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-03-23T13:21:29.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.089 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-03-23T13:21:29.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-03-23T13:21:29.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.089 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T13:21:29.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.090 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-03-23T13:21:29.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.090 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.090 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-03-23T13:21:29.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-03-23T13:21:29.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.091 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.091 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T13:21:29.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.091 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.091 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-03-23T13:21:29.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.092 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.092 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-03-23T13:21:29.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-03-23T13:21:29.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T13:21:29.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.093 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.093 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-03-23T13:21:29.093 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.094 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-03-23T13:21:29.094 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-03-23T13:21:29.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.094 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.095 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T13:21:29.095 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.095 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.095 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-03-23T13:21:29.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.096 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-03-23T13:21:29.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-03-23T13:21:29.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.096 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T13:21:29.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-03-23T13:21:29.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.097 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.097 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-03-23T13:21:29.097 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-03-23T13:21:29.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.098 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T13:21:29.098 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.098 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.099 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:29.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.099 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.099 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-03-23T13:21:29.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-03-23T13:21:29.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-23T13:21:29.100 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.100 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-03-23T13:21:29.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.101 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.101 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-03-23T13:21:29.101 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-03-23T13:21:29.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.102 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T13:21:29.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.102 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:29.103 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.103 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.103 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-03-23T13:21:29.103 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-03-23T13:21:29.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.104 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T13:21:29.104 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.104 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.104 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-03-23T13:21:29.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.105 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.105 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-03-23T13:21:29.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-03-23T13:21:29.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T13:21:29.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.106 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-03-23T13:21:29.106 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.106 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-03-23T13:21:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-03-23T13:21:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.108 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T13:21:29.108 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.108 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.108 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-03-23T13:21:29.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-03-23T13:21:29.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.109 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:29.109 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.109 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T13:21:29.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.110 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-03-23T13:21:29.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T13:21:29.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-03-23T13:21:29.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T13:21:29.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-03-23T13:21:29.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-03-23T13:21:29.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.112 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.112 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-03-23T13:21:29.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T13:21:29.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.113 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-03-23T13:21:29.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-03-23T13:21:29.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-03-23T13:21:29.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.114 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-03-23T13:21:29.114 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.115 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-03-23T13:21:29.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-03-23T13:21:29.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T13:21:29.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.116 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-03-23T13:21:29.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-03-23T13:21:29.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.117 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:29.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-03-23T13:21:29.117 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.117 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.118 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-03-23T13:21:29.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-03-23T13:21:29.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.118 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.118 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:29.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.118 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-03-23T13:21:29.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.119 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-03-23T13:21:29.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-03-23T13:21:29.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.120 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T13:21:29.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.121 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-03-23T13:21:29.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-03-23T13:21:29.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.122 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.122 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T13:21:29.122 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.122 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.122 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.122 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.123 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.123 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.123 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.123 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.123 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.124 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.124 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.124 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.124 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.124 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.124 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.125 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-03-23T13:21:29.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-03-23T13:21:29.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.125 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.126 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T13:21:29.126 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.126 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.127 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.127 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T13:21:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.129 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.129 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.129 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.129 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.129 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.129 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.129 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.130 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.130 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.130 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.130 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.130 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.131 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.131 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.131 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.131 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.131 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.132 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.132 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T13:21:29.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T13:21:29.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T13:21:29.133 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.134 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T13:21:29.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T13:21:29.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.134 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-03-23T13:21:29.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.135 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T13:21:29.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.135 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-03-23T13:21:29.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-03-23T13:21:29.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.136 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.137 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-03-23T13:21:29.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-03-23T13:21:29.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-03-23T13:21:29.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.138 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.138 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.138 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.138 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.138 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T13:21:29.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.139 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.139 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-03-23T13:21:29.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-03-23T13:21:29.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-03-23T13:21:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-23T13:21:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-23T13:21:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-03-23T13:21:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-03-23T13:21:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-03-23T13:21:29.142 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.142 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T13:21:29.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.142 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-03-23T13:21:29.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-03-23T13:21:29.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.143 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-03-23T13:21:29.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T13:21:29.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.144 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-03-23T13:21:29.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-03-23T13:21:29.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.144 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T13:21:29.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T13:21:29.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.145 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-03-23T13:21:29.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-03-23T13:21:29.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.146 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.146 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.146 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T13:21:29.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.147 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-03-23T13:21:29.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-03-23T13:21:29.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.147 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T13:21:29.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.148 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.148 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-03-23T13:21:29.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-03-23T13:21:29.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.149 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.149 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.149 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T13:21:29.150 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.150 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-03-23T13:21:29.150 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-03-23T13:21:29.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.151 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T13:21:29.152 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.152 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.152 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-03-23T13:21:29.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-03-23T13:21:29.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.153 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T13:21:29.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.154 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-03-23T13:21:29.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-03-23T13:21:29.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.155 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-03-23T13:21:29.155 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.155 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.155 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-03-23T13:21:29.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.156 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-03-23T13:21:29.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-03-23T13:21:29.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.157 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.157 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.157 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T13:21:29.157 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.158 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-03-23T13:21:29.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-03-23T13:21:29.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.158 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.159 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T13:21:29.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.159 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-03-23T13:21:29.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-03-23T13:21:29.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.160 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T13:21:29.160 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.161 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-03-23T13:21:29.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-03-23T13:21:29.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.162 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T13:21:29.162 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.163 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-03-23T13:21:29.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-03-23T13:21:29.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.163 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T13:21:29.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.164 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-03-23T13:21:29.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-03-23T13:21:29.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T13:21:29.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-03-23T13:21:29.166 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.169 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.169 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.169 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.169 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.170 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.170 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.171 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.171 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.171 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.171 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.171 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.172 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.172 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.172 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.172 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.172 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.172 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.173 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.173 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.174 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.174 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-03-23T13:21:29.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-03-23T13:21:29.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-03-23T13:21:29.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T13:21:29.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.176 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-03-23T13:21:29.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-03-23T13:21:29.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.176 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-03-23T13:21:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T13:21:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-03-23T13:21:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T13:21:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T13:21:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.180 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.180 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T13:21:29.180 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T13:21:29.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.180 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.181 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.181 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.181 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.181 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T13:21:29.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T13:21:29.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.182 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.183 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.183 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-03-23T13:21:29.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T13:21:29.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.183 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.183 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.183 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.184 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.184 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T13:21:29.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T13:21:29.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.185 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.185 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.185 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.185 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.186 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.186 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T13:21:29.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T13:21:29.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.187 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T13:21:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T13:21:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T13:21:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T13:21:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T13:21:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T13:21:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T13:21:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T13:21:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T13:21:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T13:21:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T13:21:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T13:21:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T13:21:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T13:21:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T13:21:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T13:21:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T13:21:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T13:21:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-23T13:21:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T13:21:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T13:21:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-03-23T13:21:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-03-23T13:21:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T13:21:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T13:21:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-23T13:21:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T13:21:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T13:21:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T13:21:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T13:21:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T13:21:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T13:21:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T13:21:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T13:21:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T13:21:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T13:21:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T13:21:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T13:21:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T13:21:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T13:21:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T13:21:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T13:21:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T13:21:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T13:21:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T13:21:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T13:21:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T13:21:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T13:21:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T13:21:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T13:21:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T13:21:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T13:21:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T13:21:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T13:21:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T13:21:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T13:21:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T13:21:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T13:21:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T13:21:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T13:21:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T13:21:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-03-23T13:21:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T13:21:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T13:21:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-03-23T13:21:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T13:21:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T13:21:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T13:21:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T13:21:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-23T13:21:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T13:21:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T13:21:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T13:21:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T13:21:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T13:21:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T13:21:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T13:21:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T13:21:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T13:21:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T13:21:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-03-23T13:21:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-03-23T13:21:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-03-23T13:21:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-03-23T13:21:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-03-23T13:21:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-03-23T13:21:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-03-23T13:21:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-03-23T13:21:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-03-23T13:21:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-03-23T13:21:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-23T13:21:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-23T13:21:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-03-23T13:21:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-03-23T13:21:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-03-23T13:21:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-03-23T13:21:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T13:21:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-23T13:21:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-23T13:21:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-23T13:21:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-23T13:21:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-23T13:21:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-03-23T13:21:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-23T13:21:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-23T13:21:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-03-23T13:21:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-03-23T13:21:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-03-23T13:21:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-23T13:21:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-23T13:21:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-03-23T13:21:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-03-23T13:21:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-23T13:21:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-23T13:21:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-23T13:21:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-23T13:21:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-23T13:21:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-03-23T13:21:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-03-23T13:21:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-23T13:21:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-03-23T13:21:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-03-23T13:21:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-03-23T13:21:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-03-23T13:21:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-03-23T13:21:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-03-23T13:21:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T13:21:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-03-23T13:21:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-03-23T13:21:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-03-23T13:21:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T13:21:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-03-23T13:21:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-03-23T13:21:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-03-23T13:21:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-03-23T13:21:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-03-23T13:21:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-03-23T13:21:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-23T13:21:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T13:21:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-03-23T13:21:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-03-23T13:21:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.300 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T13:21:29.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.300 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.300 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:29.300 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:29.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:29.301 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.301 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.301 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.301 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T13:21:29.302 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.302 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.302 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-03-23T13:21:29.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-03-23T13:21:29.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.302 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-03-23T13:21:29.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.303 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T13:21:29.303 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.303 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-03-23T13:21:29.304 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.304 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.304 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.304 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.305 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.305 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.305 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.305 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.306 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.306 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.306 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.306 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.307 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.307 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.308 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.308 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.308 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.309 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.309 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.309 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.309 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.309 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.309 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.309 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.310 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.310 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.310 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.310 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.310 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.310 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.310 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.311 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.311 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.312 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-03-23T13:21:29.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-03-23T13:21:29.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.312 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-23T13:21:29.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.313 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.313 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.313 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.314 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-03-23T13:21:29.314 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-03-23T13:21:29.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.314 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.314 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-03-23T13:21:29.315 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.315 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.315 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.315 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.315 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.315 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.315 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:29.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:29.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.316 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.316 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:29.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.316 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.317 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.317 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-03-23T13:21:29.317 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-03-23T13:21:29.318 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-03-23T13:21:29.318 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.318 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-03-23T13:21:29.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-03-23T13:21:29.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.318 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-03-23T13:21:29.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.319 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T13:21:29.319 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.319 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.319 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-03-23T13:21:29.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-03-23T13:21:29.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.320 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.320 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-03-23T13:21:29.320 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.320 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.320 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.320 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T13:21:29.320 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.321 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.321 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-03-23T13:21:29.321 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-03-23T13:21:29.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.322 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.322 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.322 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-03-23T13:21:29.322 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.322 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:29.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:29.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.323 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.323 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-03-23T13:21:29.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-03-23T13:21:29.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.324 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T13:21:29.324 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.324 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.324 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.324 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.325 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.325 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.325 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.325 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.325 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.325 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.325 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.326 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.326 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.326 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.326 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.326 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.327 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.327 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.328 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.328 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.328 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.329 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.329 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.329 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.329 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.329 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.330 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.330 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.330 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.330 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.330 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.330 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.330 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.331 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.331 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.331 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.331 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.331 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.331 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.332 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.332 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.332 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-03-23T13:21:29.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-03-23T13:21:29.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.333 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.333 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.333 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-03-23T13:21:29.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-03-23T13:21:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-03-23T13:21:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-03-23T13:21:29.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.335 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.335 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-03-23T13:21:29.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.335 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-23T13:21:29.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-23T13:21:29.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-03-23T13:21:29.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.336 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T13:21:29.336 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.337 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-03-23T13:21:29.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-03-23T13:21:29.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.337 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.337 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:29.337 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:29.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-03-23T13:21:29.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T13:21:29.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.338 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.338 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-03-23T13:21:29.338 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-03-23T13:21:29.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.339 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.339 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.339 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.339 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.339 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-03-23T13:21:29.339 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.340 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.340 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-03-23T13:21:29.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-03-23T13:21:29.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-03-23T13:21:29.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.341 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.341 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.341 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T13:21:29.341 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-03-23T13:21:29.342 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.342 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.342 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.342 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.342 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.343 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.343 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.343 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.343 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.343 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.344 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.344 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.344 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.344 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.345 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.345 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.345 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.345 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.346 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.346 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.346 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.346 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.347 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.347 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.347 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.347 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.348 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.348 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.348 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.348 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.348 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.348 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.348 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.349 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.349 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.350 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.350 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.350 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.350 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.351 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.351 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-03-23T13:21:29.351 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.352 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.352 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.352 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.352 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.352 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.352 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.353 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.353 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.353 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.353 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.353 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.353 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.354 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.354 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.354 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.354 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.355 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.355 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.356 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.356 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.356 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.356 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.356 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.357 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.357 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.357 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.358 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.358 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.358 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.358 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.358 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.358 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.358 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.358 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.359 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.359 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.359 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.359 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.360 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.360 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.360 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.360 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-03-23T13:21:29.361 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.361 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.362 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.362 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-03-23T13:21:29.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-03-23T13:21:29.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.362 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.362 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-03-23T13:21:29.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.363 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.363 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T13:21:29.363 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.363 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.363 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-03-23T13:21:29.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-03-23T13:21:29.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.364 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.364 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.364 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-03-23T13:21:29.364 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T13:21:29.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.365 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.365 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.365 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.365 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.366 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.366 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.366 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.366 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.366 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.366 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.366 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.366 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.367 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.367 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.367 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.367 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.368 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.368 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.368 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.368 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.369 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.369 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.369 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.369 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.370 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.370 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.370 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.370 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.370 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.370 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.370 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.370 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.371 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.372 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-03-23T13:21:29.374 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.374 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.374 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.375 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-03-23T13:21:29.375 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-03-23T13:21:29.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.375 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.375 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-03-23T13:21:29.375 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.375 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.376 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-03-23T13:21:29.376 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T13:21:29.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.376 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.376 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.377 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.377 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.377 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.377 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.377 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.377 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.377 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.378 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.378 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.378 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.378 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.378 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.378 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.379 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.379 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.379 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.379 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.379 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.380 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.380 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.380 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.381 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.381 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.382 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.382 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.382 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.382 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.382 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.382 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.382 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.383 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.383 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.383 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.384 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.384 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.384 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.384 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.384 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-03-23T13:21:29.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-03-23T13:21:29.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.386 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-03-23T13:21:29.386 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T13:21:29.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.386 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-03-23T13:21:29.386 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-03-23T13:21:29.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.387 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.387 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-03-23T13:21:29.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-03-23T13:21:29.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.388 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.388 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-03-23T13:21:29.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-03-23T13:21:29.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-03-23T13:21:29.388 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T13:21:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-03-23T13:21:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-03-23T13:21:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-03-23T13:21:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-03-23T13:21:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-03-23T13:21:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-03-23T13:21:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T13:21:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T13:21:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-03-23T13:21:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-03-23T13:21:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-03-23T13:21:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-23T13:21:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-03-23T13:21:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-03-23T13:21:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T13:21:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-03-23T13:21:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-23T13:21:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-23T13:21:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-03-23T13:21:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-03-23T13:21:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-03-23T13:21:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-03-23T13:21:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-03-23T13:21:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-03-23T13:21:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.456 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.456 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.456 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.456 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.456 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.456 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.456 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.456 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.457 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.457 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.457 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.457 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.457 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.457 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.457 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.458 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.458 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.458 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.458 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.459 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.459 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.459 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.459 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.459 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.460 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.460 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.460 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.460 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.460 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.460 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.461 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.461 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.461 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.461 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.461 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.461 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.461 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.461 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.462 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.462 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.463 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.463 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.463 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.463 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.464 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:29.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:29.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.464 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.464 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:29.464 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.465 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.465 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.465 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.465 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-03-23T13:21:29.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-03-23T13:21:29.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.466 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.466 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-23T13:21:29.466 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-03-23T13:21:29.466 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.466 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.466 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-03-23T13:21:29.467 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T13:21:29.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.467 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.467 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-03-23T13:21:29.468 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.468 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.468 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.468 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-03-23T13:21:29.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-03-23T13:21:29.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.468 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.469 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.469 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.469 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.469 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T13:21:29.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.469 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.470 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-23T13:21:29.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-23T13:21:29.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.470 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.471 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.471 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.471 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.471 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.471 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.471 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.472 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.472 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.472 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.472 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.472 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.473 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.473 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.473 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.473 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.473 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.473 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.474 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.474 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.474 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.474 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.475 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.475 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.475 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.475 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.475 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.477 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.477 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.477 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.477 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.477 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.477 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.477 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.477 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.477 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.478 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.478 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.478 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.478 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.479 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.479 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.479 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.479 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.479 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.479 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.480 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-03-23T13:21:29.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-03-23T13:21:29.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.480 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.480 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-03-23T13:21:29.480 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.481 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.481 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-03-23T13:21:29.481 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.481 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.481 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:29.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:29.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.482 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.482 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:29.482 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.482 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.482 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.482 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-03-23T13:21:29.483 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.483 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-03-23T13:21:29.483 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.484 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.484 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.484 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-03-23T13:21:29.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-03-23T13:21:29.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.484 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.485 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.485 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.485 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.485 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-03-23T13:21:29.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-03-23T13:21:29.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.486 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.486 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.486 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T13:21:29.487 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T13:21:29.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.487 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.487 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.487 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.488 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.488 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.488 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.488 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.488 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.488 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.488 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.488 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.489 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.489 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.489 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.489 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.489 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.489 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.490 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.490 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.490 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.490 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.490 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.491 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.491 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.491 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.492 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.492 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.492 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.492 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.492 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.492 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.493 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.493 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.493 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.493 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.493 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.493 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.493 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.494 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.494 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.494 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.494 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.495 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.495 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.495 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.495 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.495 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.495 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-03-23T13:21:29.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-03-23T13:21:29.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.496 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.497 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.497 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.497 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T13:21:29.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T13:21:29.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.497 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.498 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-03-23T13:21:29.498 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.498 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.498 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.498 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.498 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.498 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.499 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.499 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-03-23T13:21:29.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-03-23T13:21:29.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.499 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.499 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-03-23T13:21:29.500 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.500 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.500 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.500 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.500 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.501 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-23T13:21:29.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-23T13:21:29.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.501 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-23T13:21:29.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.502 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T13:21:29.502 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.502 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.502 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-23T13:21:29.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-23T13:21:29.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.503 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.503 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-03-23T13:21:29.503 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.503 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T13:21:29.503 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.504 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.504 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-03-23T13:21:29.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-03-23T13:21:29.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.504 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.504 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-03-23T13:21:29.505 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.505 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.505 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.505 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.505 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.505 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.505 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-23T13:21:29.506 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-23T13:21:29.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.506 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.506 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-03-23T13:21:29.506 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.506 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.506 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.507 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-03-23T13:21:29.507 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-03-23T13:21:29.507 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.507 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.507 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.507 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-23T13:21:29.508 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-23T13:21:29.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.508 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.508 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-23T13:21:29.508 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-23T13:21:29.508 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-23T13:21:29.508 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.508 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.509 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.509 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-23T13:21:29.510 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.510 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.510 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.510 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.510 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.511 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.511 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.511 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.511 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.511 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.511 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.511 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.511 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.511 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.512 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.512 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.512 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.512 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.513 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.513 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.514 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.514 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.514 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.516 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.516 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.516 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.516 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.516 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.516 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.516 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.517 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.517 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.517 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.517 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.517 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.518 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.518 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.518 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.518 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.518 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-03-23T13:21:29.518 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-03-23T13:21:29.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.519 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.519 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.519 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.519 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.519 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.519 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.519 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.520 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.520 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-03-23T13:21:29.520 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T13:21:29.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.520 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.521 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.521 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.521 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.521 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.521 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.521 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.522 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-03-23T13:21:29.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-03-23T13:21:29.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.522 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.522 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.522 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.522 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.523 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.523 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.523 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-03-23T13:21:29.524 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.524 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.524 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.524 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-03-23T13:21:29.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-03-23T13:21:29.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.524 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.525 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.525 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.525 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.525 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.525 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.526 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-23T13:21:29.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-23T13:21:29.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.526 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.526 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.526 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.527 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.527 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.527 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.527 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.527 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.528 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.528 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.528 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.528 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.528 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.528 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.528 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.529 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.529 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.529 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.529 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.529 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.529 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.529 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.529 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.530 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.530 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.530 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.530 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.530 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.531 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.531 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.531 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.532 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.532 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.532 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.532 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.532 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.532 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.532 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.533 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.533 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.533 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.533 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.533 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.533 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.533 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.533 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.533 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.534 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.534 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.534 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.534 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.534 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.535 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.535 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.535 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.535 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.535 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.535 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-23T13:21:29.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-23T13:21:29.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.536 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.536 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.536 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.536 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.537 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.537 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.537 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.537 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-23T13:21:29.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-23T13:21:29.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.538 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.538 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.538 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.538 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.538 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.538 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.539 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-03-23T13:21:29.540 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.540 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.540 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.540 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.541 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.541 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.541 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.541 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.541 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.541 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.542 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.542 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.542 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.542 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.542 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.542 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.542 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.543 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.543 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.543 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.544 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.544 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.544 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.544 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.545 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.545 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.545 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.545 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.546 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.546 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.546 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.546 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.546 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.546 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.546 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.547 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.547 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.547 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.547 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.547 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.547 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.547 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.547 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.548 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.548 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.548 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.548 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.549 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.549 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.549 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.549 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.549 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-03-23T13:21:29.550 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.550 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.550 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.550 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.551 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.551 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.551 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.551 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.551 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.551 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.553 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.556 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.556 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.556 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.556 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.556 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.556 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.556 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.556 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.557 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.557 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.557 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.557 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.558 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.558 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.558 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.558 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.558 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.558 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.559 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.559 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-03-23T13:21:29.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-03-23T13:21:29.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.559 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.559 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.559 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-03-23T13:21:29.560 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.560 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.560 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.560 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T13:21:29.560 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.560 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-03-23T13:21:29.561 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.561 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.561 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.561 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.562 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.562 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.562 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.562 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.562 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.563 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.563 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.563 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.563 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.563 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.563 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.563 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.564 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.564 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.564 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.564 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.564 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.565 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.565 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.565 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.565 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.565 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.565 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.565 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.566 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.566 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.566 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.567 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.567 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.567 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.567 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.567 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.567 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.567 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.568 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.568 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.568 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.568 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.568 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.569 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.569 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.569 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.569 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.569 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.569 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.569 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.570 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.570 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-03-23T13:21:29.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-03-23T13:21:29.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.570 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.571 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.571 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.572 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-03-23T13:21:29.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-03-23T13:21:29.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.572 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.572 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-03-23T13:21:29.572 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.573 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.573 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.573 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.573 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.573 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-03-23T13:21:29.573 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-03-23T13:21:29.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.574 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.574 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.574 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.575 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.575 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-03-23T13:21:29.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-03-23T13:21:29.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.575 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.576 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.576 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.576 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.576 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.576 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.576 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.577 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.577 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-03-23T13:21:29.577 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-03-23T13:21:29.577 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.577 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.577 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.577 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.578 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.578 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.578 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.578 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.578 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.579 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-03-23T13:21:29.579 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-03-23T13:21:29.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.579 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.579 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.579 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.579 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-03-23T13:21:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-03-23T13:21:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-03-23T13:21:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-03-23T13:21:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-03-23T13:21:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-03-23T13:21:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-03-23T13:21:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-03-23T13:21:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-03-23T13:21:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-03-23T13:21:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-03-23T13:21:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-03-23T13:21:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-03-23T13:21:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-03-23T13:21:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-03-23T13:21:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-03-23T13:21:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T13:21:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-03-23T13:21:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-03-23T13:21:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-03-23T13:21:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-03-23T13:21:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-03-23T13:21:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-03-23T13:21:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-03-23T13:21:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-03-23T13:21:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T13:21:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-03-23T13:21:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-03-23T13:21:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-03-23T13:21:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-03-23T13:21:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T13:21:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-03-23T13:21:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-03-23T13:21:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T13:21:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-03-23T13:21:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-03-23T13:21:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T13:21:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-03-23T13:21:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-03-23T13:21:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T13:21:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-03-23T13:21:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-03-23T13:21:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T13:21:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-03-23T13:21:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-03-23T13:21:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-03-23T13:21:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-03-23T13:21:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-03-23T13:21:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-03-23T13:21:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T13:21:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-23T13:21:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-23T13:21:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-03-23T13:21:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-03-23T13:21:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-03-23T13:21:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-03-23T13:21:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-03-23T13:21:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-03-23T13:21:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-03-23T13:21:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-03-23T13:21:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-03-23T13:21:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-03-23T13:21:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-03-23T13:21:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-03-23T13:21:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-03-23T13:21:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-03-23T13:21:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-03-23T13:21:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T13:21:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-03-23T13:21:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-23T13:21:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-03-23T13:21:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-03-23T13:21:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-03-23T13:21:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T13:21:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-03-23T13:21:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-03-23T13:21:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-03-23T13:21:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-03-23T13:21:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-03-23T13:21:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T13:21:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-03-23T13:21:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T13:21:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T13:21:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-03-23T13:21:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-03-23T13:21:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-03-23T13:21:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-03-23T13:21:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-03-23T13:21:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-03-23T13:21:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.703 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.703 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.703 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.703 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.703 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.703 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-03-23T13:21:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-03-23T13:21:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-03-23T13:21:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-03-23T13:21:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-03-23T13:21:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-03-23T13:21:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-03-23T13:21:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T13:21:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-23T13:21:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "modules": [ 2024-03-23T13:21:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-23T13:21:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard", 2024-03-23T13:21:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "iostat", 2024-03-23T13:21:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "nfs", 2024-03-23T13:21:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "restful" 2024-03-23T13:21:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-03-23T13:21:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-03-23T13:21:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T13:21:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-03-23T13:21:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-03-23T13:21:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-03-23T13:21:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-03-23T13:21:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-03-23T13:21:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-03-23T13:21:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-03-23T13:21:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-03-23T13:21:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-03-23T13:21:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-03-23T13:21:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-03-23T13:21:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-03-23T13:21:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-03-23T13:21:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-03-23T13:21:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-03-23T13:21:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-03-23T13:21:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-03-23T13:21:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-03-23T13:21:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-03-23T13:21:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-03-23T13:21:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-03-23T13:21:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-03-23T13:21:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T13:21:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-03-23T13:21:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-03-23T13:21:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-03-23T13:21:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-03-23T13:21:29.748 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.748 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.748 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.749 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-03-23T13:21:29.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-03-23T13:21:29.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.749 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.750 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.750 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.750 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.750 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.751 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T13:21:29.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.752 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.752 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-03-23T13:21:29.753 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-03-23T13:21:29.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.753 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.754 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-03-23T13:21:29.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.755 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T13:21:29.755 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T13:21:29.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.755 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.756 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-03-23T13:21:29.756 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-03-23T13:21:29.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:29.757 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.757 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:29.757 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-23T13:21:29.758 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-23T13:21:29.758 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.758 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T13:21:29.758 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T13:21:29.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.759 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-03-23T13:21:29.760 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-03-23T13:21:29.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:29.760 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.761 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-03-23T13:21:29.761 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:29.761 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-03-23T13:21:29.762 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.762 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T13:21:29.762 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.763 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.763 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-03-23T13:21:29.763 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-03-23T13:21:29.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.764 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.764 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T13:21:29.765 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.765 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.765 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.765 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T13:21:29.766 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T13:21:29.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.766 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.767 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-03-23T13:21:29.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-03-23T13:21:29.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.767 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.768 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-03-23T13:21:29.768 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-03-23T13:21:29.768 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-03-23T13:21:29.769 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.769 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-03-23T13:21:29.769 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T13:21:29.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.770 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.770 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-03-23T13:21:29.770 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-03-23T13:21:29.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.772 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-03-23T13:21:29.772 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.772 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.772 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.773 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T13:21:29.773 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T13:21:29.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.773 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.774 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-03-23T13:21:29.774 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-03-23T13:21:29.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:29.775 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.775 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.775 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:29.775 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-23T13:21:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-23T13:21:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T13:21:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T13:21:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-03-23T13:21:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-03-23T13:21:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T13:21:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-23T13:21:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-23T13:21:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-03-23T13:21:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-03-23T13:21:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-03-23T13:21:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-03-23T13:21:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-03-23T13:21:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-03-23T13:21:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-03-23T13:21:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-03-23T13:21:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-03-23T13:21:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T13:21:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T13:21:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-03-23T13:21:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-03-23T13:21:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T13:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T13:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-03-23T13:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-03-23T13:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T13:21:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-03-23T13:21:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-03-23T13:21:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-03-23T13:21:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-23T13:21:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T13:21:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-03-23T13:21:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-03-23T13:21:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-03-23T13:21:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T13:21:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-03-23T13:21:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-03-23T13:21:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-03-23T13:21:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T13:21:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-03-23T13:21:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-03-23T13:21:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T13:21:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T13:21:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-03-23T13:21:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-03-23T13:21:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T13:21:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-03-23T13:21:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T13:21:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-03-23T13:21:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T13:21:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-03-23T13:21:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-03-23T13:21:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-03-23T13:21:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-03-23T13:21:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-03-23T13:21:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-23T13:21:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-03-23T13:21:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-03-23T13:21:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T13:21:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-03-23T13:21:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-03-23T13:21:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T13:21:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-03-23T13:21:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-03-23T13:21:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T13:21:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-03-23T13:21:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-03-23T13:21:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T13:21:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-03-23T13:21:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-03-23T13:21:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T13:21:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-03-23T13:21:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-03-23T13:21:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-03-23T13:21:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T13:21:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-03-23T13:21:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-03-23T13:21:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-03-23T13:21:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-23T13:21:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-03-23T13:21:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-03-23T13:21:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T13:21:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-03-23T13:21:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-03-23T13:21:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-03-23T13:21:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T13:21:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-03-23T13:21:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-03-23T13:21:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-03-23T13:21:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T13:21:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-03-23T13:21:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-03-23T13:21:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-03-23T13:21:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T13:21:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-03-23T13:21:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-03-23T13:21:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-03-23T13:21:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T13:21:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-03-23T13:21:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-03-23T13:21:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-03-23T13:21:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T13:21:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-03-23T13:21:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-03-23T13:21:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-03-23T13:21:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T13:21:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-03-23T13:21:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-03-23T13:21:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-23T13:21:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-03-23T13:21:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-03-23T13:21:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-03-23T13:21:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T13:21:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-23T13:21:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-03-23T13:21:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-03-23T13:21:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T13:21:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-03-23T13:21:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-03-23T13:21:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-03-23T13:21:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T13:21:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-03-23T13:21:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-03-23T13:21:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-03-23T13:21:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T13:21:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-03-23T13:21:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-03-23T13:21:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T13:21:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-03-23T13:21:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T13:21:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-03-23T13:21:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T13:21:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-03-23T13:21:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-03-23T13:21:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-03-23T13:21:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T13:21:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-03-23T13:21:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-03-23T13:21:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-03-23T13:21:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-03-23T13:21:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-03-23T13:21:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-03-23T13:21:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T13:21:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-03-23T13:21:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-03-23T13:21:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-03-23T13:21:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-03-23T13:21:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-03-23T13:21:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-03-23T13:21:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-03-23T13:21:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-03-23T13:21:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T13:21:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-03-23T13:21:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-03-23T13:21:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T13:21:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-03-23T13:21:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-03-23T13:21:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T13:21:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T13:21:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T13:21:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T13:21:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T13:21:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T13:21:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T13:21:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-03-23T13:21:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T13:21:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-03-23T13:21:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-03-23T13:21:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-03-23T13:21:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-03-23T13:21:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-03-23T13:21:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T13:21:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-03-23T13:21:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-03-23T13:21:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-03-23T13:21:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-23T13:21:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-23T13:21:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-03-23T13:21:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-03-23T13:21:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-03-23T13:21:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T13:21:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-03-23T13:21:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-03-23T13:21:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-03-23T13:21:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T13:21:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-03-23T13:21:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-03-23T13:21:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T13:21:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T13:21:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-03-23T13:21:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-03-23T13:21:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T13:21:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-03-23T13:21:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-03-23T13:21:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T13:21:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-03-23T13:21:29.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-03-23T13:21:29.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.899 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.899 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.899 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.899 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.899 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T13:21:29.900 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.900 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.900 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-03-23T13:21:29.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-03-23T13:21:29.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.900 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.901 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.901 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.901 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.901 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.901 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T13:21:29.901 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.901 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.902 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-03-23T13:21:29.902 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-03-23T13:21:29.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.902 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.902 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.902 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.903 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.903 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.903 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T13:21:29.903 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.903 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-03-23T13:21:29.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-03-23T13:21:29.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.904 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-03-23T13:21:29.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-03-23T13:21:29.904 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.905 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-03-23T13:21:29.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-03-23T13:21:29.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.905 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.905 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.906 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.906 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T13:21:29.906 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.906 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.907 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-03-23T13:21:29.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-03-23T13:21:29.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.907 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.907 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.907 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.907 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.907 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T13:21:29.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.908 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.908 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-03-23T13:21:29.908 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-03-23T13:21:29.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.908 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.909 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.909 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.909 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T13:21:29.909 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.909 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.910 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-03-23T13:21:29.910 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-03-23T13:21:29.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.910 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.910 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.910 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.910 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.910 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T13:21:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-03-23T13:21:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-03-23T13:21:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T13:21:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-03-23T13:21:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-03-23T13:21:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T13:21:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-03-23T13:21:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-03-23T13:21:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-03-23T13:21:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-03-23T13:21:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T13:21:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-03-23T13:21:29.925 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-03-23T13:21:29.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:29.925 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:29.925 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-03-23T13:21:29.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.925 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.925 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.926 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T13:21:29.926 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.926 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.926 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:29.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:29.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-03-23T13:21:29.927 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:29.927 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:29.927 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:29.927 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T13:21:29.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T13:21:29.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.927 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.929 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T13:21:29.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T13:21:29.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.929 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.930 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.930 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.930 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.930 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.931 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T13:21:29.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T13:21:29.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.931 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.931 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.931 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.931 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.932 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.932 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-03-23T13:21:29.932 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T13:21:29.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.933 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.933 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.933 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.933 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T13:21:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T13:21:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.935 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.935 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.935 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.935 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.935 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.935 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T13:21:29.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T13:21:29.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.936 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.936 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.936 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.936 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.936 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.936 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.937 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.937 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T13:21:29.937 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T13:21:29.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T13:21:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T13:21:29.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.939 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.939 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.939 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.939 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.940 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.940 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T13:21:29.940 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T13:21:29.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.940 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.941 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.941 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.942 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T13:21:29.942 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T13:21:29.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.942 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.942 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.942 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.942 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.943 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.943 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.943 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.943 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.943 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T13:21:29.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T13:21:29.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.943 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.944 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.944 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.944 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.945 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.945 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T13:21:29.945 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T13:21:29.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.945 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.945 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.946 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.946 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.947 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T13:21:29.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T13:21:29.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.947 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.947 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.948 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.948 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.948 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.948 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T13:21:29.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T13:21:29.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.949 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.949 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.950 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T13:21:29.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T13:21:29.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.950 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-23T13:21:29.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.951 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.951 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.952 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T13:21:29.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T13:21:29.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.952 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.952 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.952 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.952 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.953 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.953 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.953 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-03-23T13:21:29.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-03-23T13:21:29.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.954 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.954 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.954 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.954 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.954 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.955 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.955 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.955 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T13:21:29.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T13:21:29.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-23T13:21:29.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.956 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.957 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T13:21:29.957 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T13:21:29.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.957 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.957 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.958 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.958 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T13:21:29.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T13:21:29.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.959 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.960 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T13:21:29.960 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T13:21:29.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.960 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.961 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.961 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.962 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.962 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T13:21:29.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T13:21:29.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.962 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.963 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.963 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.963 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.963 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.964 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T13:21:29.964 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T13:21:29.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.964 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.964 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.964 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.965 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.965 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.965 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.965 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T13:21:29.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T13:21:29.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.966 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.966 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.966 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.967 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T13:21:29.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T13:21:29.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.967 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.968 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.968 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.968 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.968 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.968 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.968 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.969 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T13:21:29.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T13:21:29.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.969 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.969 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.969 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.970 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.970 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.970 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T13:21:29.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T13:21:29.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.971 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.971 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.972 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.972 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.972 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.972 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T13:21:29.972 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T13:21:29.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.973 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.973 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.973 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.973 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.974 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.974 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T13:21:29.974 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T13:21:29.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.975 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.975 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.975 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.975 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.976 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T13:21:29.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T13:21:29.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.976 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.976 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.977 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.977 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.977 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.978 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T13:21:29.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T13:21:29.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.978 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.978 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:29.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.979 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.979 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T13:21:29.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T13:21:29.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.980 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.980 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.980 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.980 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.980 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.981 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.981 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T13:21:29.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T13:21:29.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T13:21:29.982 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.982 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.982 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.983 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T13:21:29.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T13:21:29.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.983 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.984 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.984 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.984 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.984 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.984 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.984 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.985 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T13:21:29.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T13:21:29.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-03-23T13:21:29.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.985 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.986 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.986 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.986 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T13:21:29.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T13:21:29.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.987 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.987 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-03-23T13:21:29.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.987 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.987 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.988 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.988 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T13:21:29.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T13:21:29.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.988 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.988 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.989 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.989 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.989 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.989 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T13:21:29.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T13:21:29.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-23T13:21:29.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.991 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.991 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T13:21:29.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T13:21:29.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:29.991 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:29.992 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.992 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.992 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.992 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.993 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T13:21:29.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T13:21:29.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:29.993 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.993 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:29.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.994 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.994 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.994 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T13:21:29.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T13:21:29.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.995 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.995 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:29.995 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.995 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.995 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.996 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T13:21:29.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T13:21:29.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.997 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:29.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.997 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.997 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.998 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:29.998 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T13:21:29.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T13:21:29.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:29.998 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:29.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:29.998 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:29.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:29.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:29.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:29.999 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:29.999 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:29.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:29.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:29.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-03-23T13:21:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-03-23T13:21:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.001 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.001 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-03-23T13:21:30.001 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-03-23T13:21:30.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.002 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.002 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.002 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.002 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.003 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.003 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-03-23T13:21:30.003 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-03-23T13:21:30.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.003 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.003 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.004 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.004 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.004 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-03-23T13:21:30.004 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.004 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.005 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-03-23T13:21:30.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-03-23T13:21:30.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.005 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.005 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-03-23T13:21:30.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.005 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.006 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.006 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.006 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.006 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-23T13:21:30.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-23T13:21:30.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.008 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.009 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.009 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.009 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.010 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.010 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.010 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.010 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.010 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.010 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.011 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.011 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.011 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.012 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.012 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.012 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.013 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.014 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.014 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.014 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.014 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.014 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.014 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.015 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.015 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.016 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.016 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.016 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-03-23T13:21:30.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-03-23T13:21:30.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-03-23T13:21:30.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.018 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.018 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-03-23T13:21:30.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T13:21:30.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.018 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.019 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.019 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.019 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.019 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-23T13:21:30.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-23T13:21:30.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-23T13:21:30.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.020 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.020 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.020 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.020 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.021 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.021 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-23T13:21:30.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-23T13:21:30.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.021 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.022 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-03-23T13:21:30.022 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.022 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.022 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.023 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-23T13:21:30.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-23T13:21:30.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.023 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.024 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.024 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.024 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.024 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.025 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-03-23T13:21:30.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-03-23T13:21:30.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.025 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.025 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-03-23T13:21:30.026 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.026 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.026 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.026 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-23T13:21:30.027 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-23T13:21:30.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-03-23T13:21:30.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-03-23T13:21:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-23T13:21:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-23T13:21:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-23T13:21:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-23T13:21:30.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-23T13:21:30.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.030 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.030 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-03-23T13:21:30.030 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-03-23T13:21:30.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.031 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.031 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.032 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.032 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-23T13:21:30.032 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.033 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.033 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.033 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-03-23T13:21:30.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-03-23T13:21:30.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.033 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.034 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.034 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.034 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-03-23T13:21:30.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.034 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.034 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.036 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.036 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.036 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.036 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.036 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.036 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.036 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.036 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.037 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.040 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.042 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-03-23T13:21:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-03-23T13:21:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.043 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-03-23T13:21:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T13:21:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-03-23T13:21:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-03-23T13:21:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-03-23T13:21:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T13:21:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-03-23T13:21:30.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-03-23T13:21:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-03-23T13:21:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-03-23T13:21:30.047 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-03-23T13:21:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-03-23T13:21:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-23T13:21:30.048 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T13:21:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-03-23T13:21:30.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-03-23T13:21:30.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.051 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T13:21:30.051 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.051 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.051 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:30.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:30.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:30.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:30.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T13:21:30.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.053 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-03-23T13:21:30.053 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-03-23T13:21:30.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:30.053 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-03-23T13:21:30.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.054 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.054 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.054 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T13:21:30.054 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.055 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-03-23T13:21:30.055 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.055 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.055 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.055 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.056 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.056 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.056 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.056 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.057 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.057 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.057 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.057 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.057 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.057 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.057 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.058 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.058 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.058 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.059 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.059 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.060 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.060 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.060 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.060 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.060 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.061 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.061 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.061 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.061 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.061 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.061 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.062 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.064 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-03-23T13:21:30.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-03-23T13:21:30.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-23T13:21:30.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.065 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-03-23T13:21:30.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-03-23T13:21:30.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-03-23T13:21:30.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.066 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.066 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-23T13:21:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.068 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.068 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.068 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-03-23T13:21:30.069 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-03-23T13:21:30.069 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-03-23T13:21:30.069 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.069 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-03-23T13:21:30.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-03-23T13:21:30.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-03-23T13:21:30.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T13:21:30.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.071 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-03-23T13:21:30.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-03-23T13:21:30.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.071 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.071 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-03-23T13:21:30.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T13:21:30.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.072 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.072 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-03-23T13:21:30.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-03-23T13:21:30.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.073 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.073 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.073 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.073 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-03-23T13:21:30.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.074 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.074 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:30.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:30.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:30.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-03-23T13:21:30.075 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-03-23T13:21:30.075 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T13:21:30.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.076 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.078 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.078 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.078 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.078 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.078 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.078 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.078 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.079 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.079 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.079 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.080 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.080 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.081 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.081 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.081 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.082 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.082 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.082 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.082 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.082 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.083 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.083 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.083 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.083 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.084 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.084 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.084 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.084 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.084 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.085 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.085 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.085 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.085 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.085 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-03-23T13:21:30.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-03-23T13:21:30.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.086 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.086 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-03-23T13:21:30.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.087 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.087 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-03-23T13:21:30.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-03-23T13:21:30.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.088 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-03-23T13:21:30.088 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-03-23T13:21:30.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.089 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-23T13:21:30.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-23T13:21:30.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.089 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-03-23T13:21:30.090 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.090 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T13:21:30.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.090 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-03-23T13:21:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-03-23T13:21:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-03-23T13:21:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T13:21:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-03-23T13:21:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-03-23T13:21:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-03-23T13:21:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-03-23T13:21:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-03-23T13:21:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-03-23T13:21:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T13:21:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-03-23T13:21:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-03-23T13:21:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-03-23T13:21:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-03-23T13:21:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-03-23T13:21:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-03-23T13:21:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T13:21:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-03-23T13:21:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-03-23T13:21:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-03-23T13:21:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T13:21:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-03-23T13:21:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-03-23T13:21:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-03-23T13:21:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-03-23T13:21:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-03-23T13:21:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T13:21:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-03-23T13:21:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-03-23T13:21:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-03-23T13:21:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T13:21:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-03-23T13:21:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-03-23T13:21:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-03-23T13:21:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-03-23T13:21:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-03-23T13:21:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-03-23T13:21:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-03-23T13:21:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T13:21:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-03-23T13:21:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-03-23T13:21:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-03-23T13:21:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-03-23T13:21:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-03-23T13:21:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-03-23T13:21:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T13:21:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T13:21:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-03-23T13:21:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-03-23T13:21:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-03-23T13:21:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-23T13:21:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-03-23T13:21:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-03-23T13:21:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T13:21:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-03-23T13:21:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-23T13:21:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-23T13:21:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-03-23T13:21:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-03-23T13:21:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-03-23T13:21:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-03-23T13:21:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-03-23T13:21:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-03-23T13:21:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-03-23T13:21:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-03-23T13:21:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-23T13:21:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-03-23T13:21:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-03-23T13:21:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T13:21:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-03-23T13:21:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-03-23T13:21:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-03-23T13:21:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T13:21:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-23T13:21:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-23T13:21:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-03-23T13:21:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-03-23T13:21:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-03-23T13:21:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-03-23T13:21:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-23T13:21:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-23T13:21:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-23T13:21:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-03-23T13:21:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-03-23T13:21:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-03-23T13:21:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-03-23T13:21:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-03-23T13:21:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-03-23T13:21:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T13:21:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T13:21:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-03-23T13:21:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-03-23T13:21:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T13:21:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T13:21:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-03-23T13:21:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-03-23T13:21:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-03-23T13:21:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-03-23T13:21:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-23T13:21:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-23T13:21:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-23T13:21:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T13:21:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-23T13:21:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-23T13:21:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-03-23T13:21:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T13:21:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-03-23T13:21:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-03-23T13:21:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-03-23T13:21:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-23T13:21:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-23T13:21:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-03-23T13:21:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-03-23T13:21:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-03-23T13:21:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-23T13:21:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-23T13:21:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-23T13:21:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-23T13:21:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-23T13:21:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-23T13:21:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-03-23T13:21:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-03-23T13:21:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-23T13:21:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-03-23T13:21:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T13:21:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-03-23T13:21:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-03-23T13:21:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-03-23T13:21:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-03-23T13:21:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-03-23T13:21:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-23T13:21:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-23T13:21:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-23T13:21:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-23T13:21:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-23T13:21:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-23T13:21:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-03-23T13:21:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-03-23T13:21:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-03-23T13:21:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-03-23T13:21:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-03-23T13:21:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T13:21:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-03-23T13:21:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-03-23T13:21:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-03-23T13:21:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-03-23T13:21:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-03-23T13:21:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-03-23T13:21:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-03-23T13:21:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-03-23T13:21:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-03-23T13:21:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-03-23T13:21:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-03-23T13:21:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-03-23T13:21:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-23T13:21:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-03-23T13:21:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-03-23T13:21:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-03-23T13:21:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-03-23T13:21:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-03-23T13:21:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-03-23T13:21:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-03-23T13:21:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-03-23T13:21:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-23T13:21:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-03-23T13:21:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-03-23T13:21:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-03-23T13:21:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-03-23T13:21:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-03-23T13:21:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-03-23T13:21:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-03-23T13:21:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-03-23T13:21:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-03-23T13:21:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-03-23T13:21:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T13:21:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-03-23T13:21:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-03-23T13:21:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-03-23T13:21:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-03-23T13:21:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-03-23T13:21:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-03-23T13:21:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-03-23T13:21:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-03-23T13:21:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T13:21:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-03-23T13:21:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-03-23T13:21:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-03-23T13:21:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-03-23T13:21:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T13:21:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-03-23T13:21:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-03-23T13:21:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T13:21:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-03-23T13:21:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-03-23T13:21:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T13:21:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-03-23T13:21:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-03-23T13:21:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T13:21:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-03-23T13:21:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-03-23T13:21:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T13:21:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-03-23T13:21:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-03-23T13:21:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-03-23T13:21:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-03-23T13:21:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.400 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-03-23T13:21:30.400 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-03-23T13:21:30.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.400 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.400 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T13:21:30.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.400 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-23T13:21:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-23T13:21:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-03-23T13:21:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-03-23T13:21:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-03-23T13:21:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-03-23T13:21:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-03-23T13:21:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-03-23T13:21:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-03-23T13:21:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-03-23T13:21:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-03-23T13:21:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-03-23T13:21:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-03-23T13:21:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-03-23T13:21:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-03-23T13:21:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-03-23T13:21:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-03-23T13:21:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T13:21:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-03-23T13:21:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-23T13:21:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-03-23T13:21:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-03-23T13:21:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-03-23T13:21:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T13:21:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-03-23T13:21:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-03-23T13:21:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-03-23T13:21:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-03-23T13:21:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-03-23T13:21:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-23T13:21:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T13:21:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-03-23T13:21:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T13:21:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-23T13:21:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T13:21:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-03-23T13:21:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-23T13:21:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-23T13:21:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-23T13:21:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-03-23T13:21:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-03-23T13:21:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-23T13:21:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-03-23T13:21:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-03-23T13:21:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-03-23T13:21:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-23T13:21:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-23T13:21:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-23T13:21:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-23T13:21:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-23T13:21:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-23T13:21:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-23T13:21:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-23T13:21:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-23T13:21:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-23T13:21:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-23T13:21:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-23T13:21:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-23T13:21:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-23T13:21:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-23T13:21:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-23T13:21:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-23T13:21:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-23T13:21:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-23T13:21:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-23T13:21:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-23T13:21:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-23T13:21:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-23T13:21:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-03-23T13:21:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-03-23T13:21:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-23T13:21:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-03-23T13:21:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-03-23T13:21:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-23T13:21:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-03-23T13:21:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-03-23T13:21:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-03-23T13:21:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-23T13:21:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-23T13:21:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-23T13:21:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T13:21:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-23T13:21:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-23T13:21:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-23T13:21:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-23T13:21:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-23T13:21:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-23T13:21:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-23T13:21:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "services": { 2024-03-23T13:21:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard": "https://172.21.15.7:8443/" 2024-03-23T13:21:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "always_on_modules": { 2024-03-23T13:21:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus": [ 2024-03-23T13:21:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-23T13:21:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-23T13:21:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-23T13:21:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-23T13:21:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-23T13:21:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-23T13:21:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-23T13:21:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-23T13:21:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-23T13:21:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-23T13:21:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific": [ 2024-03-23T13:21:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-23T13:21:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-23T13:21:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-23T13:21:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-23T13:21:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-23T13:21:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-23T13:21:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-23T13:21:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-23T13:21:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-23T13:21:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-23T13:21:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy": [ 2024-03-23T13:21:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-23T13:21:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-23T13:21:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-23T13:21:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-23T13:21:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-23T13:21:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-23T13:21:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-23T13:21:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-23T13:21:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-23T13:21:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-23T13:21:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "reef": [ 2024-03-23T13:21:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-23T13:21:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-23T13:21:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-23T13:21:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-23T13:21:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-23T13:21:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-23T13:21:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-23T13:21:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-23T13:21:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-23T13:21:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-23T13:21:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-23T13:21:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "squid": [ 2024-03-23T13:21:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-23T13:21:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-23T13:21:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-23T13:21:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-23T13:21:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-23T13:21:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-23T13:21:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-23T13:21:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-23T13:21:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-23T13:21:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-23T13:21:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-23T13:21:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "last_failure_osd_epoch": 3, 2024-03-23T13:21:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "active_clients": [ 2024-03-23T13:21:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-23T13:21:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-23T13:21:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-23T13:21:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-23T13:21:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 797831695 2024-03-23T13:21:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-23T13:21:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "libcephsqlite", 2024-03-23T13:21:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-23T13:21:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-23T13:21:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-23T13:21:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1567023463 2024-03-23T13:21:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-23T13:21:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-23T13:21:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-23T13:21:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-23T13:21:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-23T13:21:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162420134 2024-03-23T13:21:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-23T13:21:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-23T13:21:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-23T13:21:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-23T13:21:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-23T13:21:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-23T13:21:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2108116369 2024-03-23T13:21:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-23T13:21:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-23T13:21:30.478 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:21:30.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 -- 172.21.15.7:0/4135422947 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc2e80a20c0 msgr2=0x7fc2e80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:30.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc2e80a20c0 0x7fc2e80a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc100e10 tx=0x7fc2f4005f90 comp rx=0 tx=0).stop 2024-03-23T13:21:30.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 -- 172.21.15.7:0/4135422947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc100fb0 msgr2=0x7fc2fc102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:21:30.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc100fb0 0x7fc2fc102460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00ea20 tx=0x7fc2ec00ee20 comp rx=0 tx=0).stop 2024-03-23T13:21:30.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 -- 172.21.15.7:0/4135422947 shutdown_connections 2024-03-23T13:21:30.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fc2e80a20c0 0x7fc2e80a4580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:30.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2fc101970 0x7fc2fc1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:30.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc100fb0 0x7fc2fc102460 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:30.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 --2- 172.21.15.7:0/4135422947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2fc0ffdc0 0x7fc2fc103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:21:30.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 -- 172.21.15.7:0/4135422947 >> 172.21.15.7:0/4135422947 conn(0x7fc2fc0fb330 msgr2=0x7fc2fc1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:21:30.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 -- 172.21.15.7:0/4135422947 shutdown_connections 2024-03-23T13:21:30.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:21:28.274+0000 7fc304747640 1 -- 172.21.15.7:0/4135422947 wait complete. 2024-03-23T13:21:30.481 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:28 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/640611628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:21:30.481 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:28 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3182867207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:21:30.481 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:28 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/255391585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:21:30.482 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:28 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/640611628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:21:30.482 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:28 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3182867207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:21:30.482 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:28 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/255391585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:21:30.482 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:29 smithi155 ceph-mon[36157]: pgmap v400: 97 pgs: 97 active+clean; 43 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 378 KiB/s wr, 76 op/s 2024-03-23T13:21:30.482 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:29 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/4135422947' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T13:21:30.483 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:28 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/640611628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:21:30.485 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:28 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3182867207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:21:30.485 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:28 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/255391585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:21:30.485 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:29 smithi130 ceph-mon[36225]: pgmap v400: 97 pgs: 97 active+clean; 43 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 378 KiB/s wr, 76 op/s 2024-03-23T13:21:30.485 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:29 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/4135422947' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T13:21:30.491 DEBUG:tasks.check_counter:Getting stats from a 2024-03-23T13:21:30.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-23T13:21:30.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:29 smithi007 ceph-mon[32184]: pgmap v400: 97 pgs: 97 active+clean; 43 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 378 KiB/s wr, 76 op/s 2024-03-23T13:21:30.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:29 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4135422947' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T13:21:30.751 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:21:31.123 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:31 smithi007 ceph-mon[32184]: pgmap v401: 97 pgs: 97 active+clean; 43 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-23T13:21:31.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:31 smithi130 ceph-mon[36225]: pgmap v401: 97 pgs: 97 active+clean; 43 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-23T13:21:31.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:31 smithi155 ceph-mon[36157]: pgmap v401: 97 pgs: 97 active+clean; 43 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-23T13:21:32.556 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:21:32.556 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:21:32.556 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 472, 2024-03-23T13:21:32.557 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 472, 2024-03-23T13:21:32.557 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 4313078, 2024-03-23T13:21:32.557 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 145997, 2024-03-23T13:21:32.557 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 6, 2024-03-23T13:21:32.557 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-23T13:21:32.557 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.041250888, 2024-03-23T13:21:32.557 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.012926823, 2024-03-23T13:21:32.557 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.012798806, 2024-03-23T13:21:32.557 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.011410262, 2024-03-23T13:21:32.558 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:32.558 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 471, 2024-03-23T13:21:32.558 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.017518305, 2024-03-23T13:21:32.558 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000037193 2024-03-23T13:21:32.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.558 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:32.558 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-23T13:21:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 448 2024-03-23T13:21:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:21:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 115, 2024-03-23T13:21:32.560 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 165, 2024-03-23T13:21:32.560 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 28856, 2024-03-23T13:21:32.560 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 204050, 2024-03-23T13:21:32.560 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 5, 2024-03-23T13:21:32.560 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 4, 2024-03-23T13:21:32.560 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.014547206, 2024-03-23T13:21:32.560 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.005413577, 2024-03-23T13:21:32.560 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.003749285, 2024-03-23T13:21:32.560 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.003692961, 2024-03-23T13:21:32.561 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:32.561 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 163, 2024-03-23T13:21:32.561 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.007557871, 2024-03-23T13:21:32.561 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000046367 2024-03-23T13:21:32.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.561 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:32.561 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.561 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.562 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.562 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T13:21:32.562 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 168528 2024-03-23T13:21:32.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.562 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:21:32.562 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 83, 2024-03-23T13:21:32.562 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 76, 2024-03-23T13:21:32.562 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 476300, 2024-03-23T13:21:32.563 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 42007, 2024-03-23T13:21:32.563 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 3, 2024-03-23T13:21:32.563 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-23T13:21:32.563 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.012512779, 2024-03-23T13:21:32.563 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.004036391, 2024-03-23T13:21:32.563 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.009912991, 2024-03-23T13:21:32.563 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.001201981, 2024-03-23T13:21:32.563 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:32.563 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 75, 2024-03-23T13:21:32.564 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.004367853, 2024-03-23T13:21:32.564 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000058238 2024-03-23T13:21:32.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.564 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:32.564 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.564 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.564 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.564 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 476032, 2024-03-23T13:21:32.565 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 41696 2024-03-23T13:21:32.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.565 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-23T13:21:32.565 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 1, 2024-03-23T13:21:32.565 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-23T13:21:32.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.565 INFO:teuthology.orchestra.run.smithi007.stdout: "finisher-MDSRank": { 2024-03-23T13:21:32.565 INFO:teuthology.orchestra.run.smithi007.stdout: "queue_len": 0, 2024-03-23T13:21:32.565 INFO:teuthology.orchestra.run.smithi007.stdout: "complete_latency": { 2024-03-23T13:21:32.565 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 1032, 2024-03-23T13:21:32.566 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.026264687, 2024-03-23T13:21:32.566 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000025450 2024-03-23T13:21:32.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:32.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.566 INFO:teuthology.orchestra.run.smithi007.stdout: "finisher-PurgeQueue": { 2024-03-23T13:21:32.566 INFO:teuthology.orchestra.run.smithi007.stdout: "queue_len": 0, 2024-03-23T13:21:32.566 INFO:teuthology.orchestra.run.smithi007.stdout: "complete_latency": { 2024-03-23T13:21:32.566 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-23T13:21:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "request": 0, 2024-03-23T13:21:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "reply": 0, 2024-03-23T13:21:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "reply_latency": { 2024-03-23T13:21:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.568 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.568 INFO:teuthology.orchestra.run.smithi007.stdout: "slow_reply": 0, 2024-03-23T13:21:32.568 INFO:teuthology.orchestra.run.smithi007.stdout: "forward": 0, 2024-03-23T13:21:32.568 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_fetch_complete": 0, 2024-03-23T13:21:32.568 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_fetch_keys": 0, 2024-03-23T13:21:32.568 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_commit": 0, 2024-03-23T13:21:32.568 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_split": 0, 2024-03-23T13:21:32.568 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_merge": 0, 2024-03-23T13:21:32.569 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes": 1583, 2024-03-23T13:21:32.569 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_top": 133, 2024-03-23T13:21:32.569 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_bottom": 1450, 2024-03-23T13:21:32.569 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_pin_tail": 0, 2024-03-23T13:21:32.569 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_pinned": 1392, 2024-03-23T13:21:32.569 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_expired": 0, 2024-03-23T13:21:32.569 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_with_caps": 0, 2024-03-23T13:21:32.569 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": 0, 2024-03-23T13:21:32.570 INFO:teuthology.orchestra.run.smithi007.stdout: "subtrees": 62, 2024-03-23T13:21:32.570 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse": 0, 2024-03-23T13:21:32.570 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_hit": 0, 2024-03-23T13:21:32.570 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_forward": 0, 2024-03-23T13:21:32.570 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_discover": 0, 2024-03-23T13:21:32.570 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_dir_fetch": 0, 2024-03-23T13:21:32.570 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_remote_ino": 0, 2024-03-23T13:21:32.570 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_lock": 0, 2024-03-23T13:21:32.571 INFO:teuthology.orchestra.run.smithi007.stdout: "load_cent": 0, 2024-03-23T13:21:32.571 INFO:teuthology.orchestra.run.smithi007.stdout: "q": 0, 2024-03-23T13:21:32.571 INFO:teuthology.orchestra.run.smithi007.stdout: "exported": 0, 2024-03-23T13:21:32.571 INFO:teuthology.orchestra.run.smithi007.stdout: "exported_inodes": 0, 2024-03-23T13:21:32.571 INFO:teuthology.orchestra.run.smithi007.stdout: "imported": 0, 2024-03-23T13:21:32.571 INFO:teuthology.orchestra.run.smithi007.stdout: "imported_inodes": 0, 2024-03-23T13:21:32.571 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_dir_fetch": 0, 2024-03-23T13:21:32.571 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_backtrace_fetch": 0, 2024-03-23T13:21:32.571 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_peer_discover": 0, 2024-03-23T13:21:32.571 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rfiles": 0, 2024-03-23T13:21:32.572 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rbytes": 0, 2024-03-23T13:21:32.572 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rsnaps": 0, 2024-03-23T13:21:32.572 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T13:21:32.572 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_set_tag": 0, 2024-03-23T13:21:32.572 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T13:21:32.572 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_inotable_repaired": 0, 2024-03-23T13:21:32.572 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dir_inodes": 0, 2024-03-23T13:21:32.572 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T13:21:32.572 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T13:21:32.573 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_file_inodes": 0, 2024-03-23T13:21:32.573 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_inode_file_caps": 0, 2024-03-23T13:21:32.573 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T13:21:32.573 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_grant": 0, 2024-03-23T13:21:32.573 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T13:21:32.573 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T13:21:32.573 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T13:21:32.573 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_caps": 0, 2024-03-23T13:21:32.573 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_caps_dirty": 0, 2024-03-23T13:21:32.574 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_cap_release": 0, 2024-03-23T13:21:32.574 INFO:teuthology.orchestra.run.smithi007.stdout: "process_request_cap_release": 0 2024-03-23T13:21:32.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.574 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache": { 2024-03-23T13:21:32.574 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_update": 0, 2024-03-23T13:21:32.574 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_update_receipt": 0, 2024-03-23T13:21:32.574 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_try_discover": 0, 2024-03-23T13:21:32.574 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_send_discover": 0, 2024-03-23T13:21:32.575 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_handle_discover": 0, 2024-03-23T13:21:32.575 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays": 0, 2024-03-23T13:21:32.575 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays_delayed": 0, 2024-03-23T13:21:32.575 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays_enqueuing": 0, 2024-03-23T13:21:32.575 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_created": 0, 2024-03-23T13:21:32.575 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_enqueued": 0, 2024-03-23T13:21:32.575 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_reintegrated": 0, 2024-03-23T13:21:32.575 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_migrated": 0, 2024-03-23T13:21:32.575 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_processing": 0, 2024-03-23T13:21:32.575 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_enqueued": 0, 2024-03-23T13:21:32.576 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_prioritized": 0, 2024-03-23T13:21:32.576 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_started": 0, 2024-03-23T13:21:32.576 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_completed": 0, 2024-03-23T13:21:32.576 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_quiesce_path": 0, 2024-03-23T13:21:32.576 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_quiesce_inode": 0, 2024-03-23T13:21:32.576 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T13:21:32.576 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_exportdir": 0, 2024-03-23T13:21:32.576 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_flush": 0, 2024-03-23T13:21:32.576 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_fragmentdir": 0, 2024-03-23T13:21:32.577 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_fragstats": 0, 2024-03-23T13:21:32.577 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_inodestats": 0 2024-03-23T13:21:32.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.577 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log": { 2024-03-23T13:21:32.577 INFO:teuthology.orchestra.run.smithi007.stdout: "evlrg": 0, 2024-03-23T13:21:32.577 INFO:teuthology.orchestra.run.smithi007.stdout: "evadd": 0, 2024-03-23T13:21:32.577 INFO:teuthology.orchestra.run.smithi007.stdout: "evex": 0, 2024-03-23T13:21:32.577 INFO:teuthology.orchestra.run.smithi007.stdout: "evtrm": 0, 2024-03-23T13:21:32.577 INFO:teuthology.orchestra.run.smithi007.stdout: "ev": 2551, 2024-03-23T13:21:32.578 INFO:teuthology.orchestra.run.smithi007.stdout: "evexg": 0, 2024-03-23T13:21:32.578 INFO:teuthology.orchestra.run.smithi007.stdout: "evexd": 0, 2024-03-23T13:21:32.578 INFO:teuthology.orchestra.run.smithi007.stdout: "segadd": 0, 2024-03-23T13:21:32.578 INFO:teuthology.orchestra.run.smithi007.stdout: "segex": 0, 2024-03-23T13:21:32.578 INFO:teuthology.orchestra.run.smithi007.stdout: "segtrm": 0, 2024-03-23T13:21:32.578 INFO:teuthology.orchestra.run.smithi007.stdout: "seg": 4, 2024-03-23T13:21:32.578 INFO:teuthology.orchestra.run.smithi007.stdout: "segmjr": 2, 2024-03-23T13:21:32.578 INFO:teuthology.orchestra.run.smithi007.stdout: "segexg": 0, 2024-03-23T13:21:32.579 INFO:teuthology.orchestra.run.smithi007.stdout: "segexd": 0, 2024-03-23T13:21:32.579 INFO:teuthology.orchestra.run.smithi007.stdout: "expos": 4194304, 2024-03-23T13:21:32.579 INFO:teuthology.orchestra.run.smithi007.stdout: "wrpos": 8365483, 2024-03-23T13:21:32.579 INFO:teuthology.orchestra.run.smithi007.stdout: "rdpos": 8363787, 2024-03-23T13:21:32.579 INFO:teuthology.orchestra.run.smithi007.stdout: "jlat": { 2024-03-23T13:21:32.579 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.579 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.579 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.580 INFO:teuthology.orchestra.run.smithi007.stdout: "replayed": 2551 2024-03-23T13:21:32.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.580 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_mem": { 2024-03-23T13:21:32.580 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 973, 2024-03-23T13:21:32.580 INFO:teuthology.orchestra.run.smithi007.stdout: "ino+": 1388, 2024-03-23T13:21:32.580 INFO:teuthology.orchestra.run.smithi007.stdout: "ino-": 415, 2024-03-23T13:21:32.580 INFO:teuthology.orchestra.run.smithi007.stdout: "dir": 77, 2024-03-23T13:21:32.580 INFO:teuthology.orchestra.run.smithi007.stdout: "dir+": 77, 2024-03-23T13:21:32.581 INFO:teuthology.orchestra.run.smithi007.stdout: "dir-": 0, 2024-03-23T13:21:32.581 INFO:teuthology.orchestra.run.smithi007.stdout: "dn": 1583, 2024-03-23T13:21:32.581 INFO:teuthology.orchestra.run.smithi007.stdout: "dn+": 1583, 2024-03-23T13:21:32.581 INFO:teuthology.orchestra.run.smithi007.stdout: "dn-": 0, 2024-03-23T13:21:32.581 INFO:teuthology.orchestra.run.smithi007.stdout: "cap": 0, 2024-03-23T13:21:32.581 INFO:teuthology.orchestra.run.smithi007.stdout: "cap+": 0, 2024-03-23T13:21:32.581 INFO:teuthology.orchestra.run.smithi007.stdout: "cap-": 0, 2024-03-23T13:21:32.581 INFO:teuthology.orchestra.run.smithi007.stdout: "rss": 54112, 2024-03-23T13:21:32.582 INFO:teuthology.orchestra.run.smithi007.stdout: "heap": 215420 2024-03-23T13:21:32.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.582 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server": { 2024-03-23T13:21:32.582 INFO:teuthology.orchestra.run.smithi007.stdout: "dispatch_client_request": 0, 2024-03-23T13:21:32.582 INFO:teuthology.orchestra.run.smithi007.stdout: "dispatch_server_request": 0, 2024-03-23T13:21:32.582 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_request": 0, 2024-03-23T13:21:32.582 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_session": 0, 2024-03-23T13:21:32.582 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_peer_request": 0, 2024-03-23T13:21:32.582 INFO:teuthology.orchestra.run.smithi007.stdout: "req_create_latency": { 2024-03-23T13:21:32.583 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.583 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.583 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.583 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getattr_latency": { 2024-03-23T13:21:32.583 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.583 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.583 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.584 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getfilelock_latency": { 2024-03-23T13:21:32.584 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.584 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.584 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.584 INFO:teuthology.orchestra.run.smithi007.stdout: "req_link_latency": { 2024-03-23T13:21:32.584 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.584 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.585 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.585 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookup_latency": { 2024-03-23T13:21:32.585 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.585 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.585 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.585 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookuphash_latency": { 2024-03-23T13:21:32.585 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.586 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.586 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.586 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupino_latency": { 2024-03-23T13:21:32.586 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.586 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.586 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.587 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupname_latency": { 2024-03-23T13:21:32.587 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.587 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.587 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.587 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupparent_latency": { 2024-03-23T13:21:32.587 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.587 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.587 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.588 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupsnap_latency": { 2024-03-23T13:21:32.588 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.588 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.588 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.588 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lssnap_latency": { 2024-03-23T13:21:32.588 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.588 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.589 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.589 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mkdir_latency": { 2024-03-23T13:21:32.589 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.590 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.590 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.590 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mknod_latency": { 2024-03-23T13:21:32.590 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.590 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.591 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.591 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mksnap_latency": { 2024-03-23T13:21:32.591 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.591 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.591 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.591 INFO:teuthology.orchestra.run.smithi007.stdout: "req_open_latency": { 2024-03-23T13:21:32.592 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.592 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.592 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.592 INFO:teuthology.orchestra.run.smithi007.stdout: "req_readdir_latency": { 2024-03-23T13:21:32.592 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.592 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.593 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.593 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rename_latency": { 2024-03-23T13:21:32.593 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.593 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.593 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.593 INFO:teuthology.orchestra.run.smithi007.stdout: "req_renamesnap_latency": { 2024-03-23T13:21:32.593 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.594 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.594 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.594 INFO:teuthology.orchestra.run.smithi007.stdout: "req_snapdiff_latency": { 2024-03-23T13:21:32.594 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.594 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.594 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.595 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmdir_latency": { 2024-03-23T13:21:32.595 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.595 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.595 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.595 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmsnap_latency": { 2024-03-23T13:21:32.595 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.595 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.595 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.596 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmxattr_latency": { 2024-03-23T13:21:32.596 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.596 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.596 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.596 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setattr_latency": { 2024-03-23T13:21:32.596 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.597 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.597 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.597 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setdirlayout_latency": { 2024-03-23T13:21:32.597 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.597 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.597 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.597 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setfilelock_latency": { 2024-03-23T13:21:32.597 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.598 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.598 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.598 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setlayout_latency": { 2024-03-23T13:21:32.598 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.598 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.598 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.599 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setxattr_latency": { 2024-03-23T13:21:32.599 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.599 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.599 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.599 INFO:teuthology.orchestra.run.smithi007.stdout: "req_symlink_latency": { 2024-03-23T13:21:32.599 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.599 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.599 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.600 INFO:teuthology.orchestra.run.smithi007.stdout: "req_unlink_latency": { 2024-03-23T13:21:32.600 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.600 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.600 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.600 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_revoke_eviction": 0, 2024-03-23T13:21:32.600 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_acquisition_throttle": 0, 2024-03-23T13:21:32.600 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getvxattr_latency": { 2024-03-23T13:21:32.600 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.601 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.601 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:32.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.601 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sessions": { 2024-03-23T13:21:32.601 INFO:teuthology.orchestra.run.smithi007.stdout: "session_count": 1, 2024-03-23T13:21:32.601 INFO:teuthology.orchestra.run.smithi007.stdout: "session_add": 1, 2024-03-23T13:21:32.601 INFO:teuthology.orchestra.run.smithi007.stdout: "session_remove": 0, 2024-03-23T13:21:32.601 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions_open": 1, 2024-03-23T13:21:32.602 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions_stale": 0, 2024-03-23T13:21:32.602 INFO:teuthology.orchestra.run.smithi007.stdout: "total_load": 0, 2024-03-23T13:21:32.602 INFO:teuthology.orchestra.run.smithi007.stdout: "average_load": 0, 2024-03-23T13:21:32.602 INFO:teuthology.orchestra.run.smithi007.stdout: "avg_session_uptime": 1207, 2024-03-23T13:21:32.602 INFO:teuthology.orchestra.run.smithi007.stdout: "mdthresh_evicted": 0 2024-03-23T13:21:32.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.602 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-23T13:21:32.602 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:21:32.603 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-23T13:21:32.603 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:21:32.603 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:21:32.603 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:21:32.603 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:21:32.603 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:21:32.603 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:21:32.603 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:21:32.603 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:21:32.604 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:21:32.604 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:21:32.604 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:21:32.604 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:21:32.604 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:21:32.604 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-23T13:21:32.604 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:21:32.604 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-23T13:21:32.605 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:21:32.605 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:21:32.605 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:21:32.605 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:21:32.605 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:21:32.605 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:21:32.605 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:21:32.605 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-23T13:21:32.605 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:21:32.606 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:21:32.606 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:21:32.606 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-23T13:21:32.606 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-23T13:21:32.606 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-23T13:21:32.606 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:21:32.606 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:21:32.606 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:21:32.607 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:21:32.607 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 224947, 2024-03-23T13:21:32.607 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 1013, 2024-03-23T13:21:32.607 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-23T13:21:32.607 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-23T13:21:32.607 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-23T13:21:32.607 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-23T13:21:32.607 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:21:32.607 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-23T13:21:32.608 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:21:32.608 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-23T13:21:32.608 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 7528, 2024-03-23T13:21:32.608 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 158, 2024-03-23T13:21:32.608 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:21:32.608 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:21:32.608 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-23T13:21:32.608 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-23T13:21:32.608 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 3425150, 2024-03-23T13:21:32.609 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 28015, 2024-03-23T13:21:32.609 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-23T13:21:32.609 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-23T13:21:32.609 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-23T13:21:32.609 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-23T13:21:32.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.609 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter": { 2024-03-23T13:21:32.609 INFO:teuthology.orchestra.run.smithi007.stdout: "op_active": 0, 2024-03-23T13:21:32.610 INFO:teuthology.orchestra.run.smithi007.stdout: "op_laggy": 0, 2024-03-23T13:21:32.610 INFO:teuthology.orchestra.run.smithi007.stdout: "op_send": 583, 2024-03-23T13:21:32.610 INFO:teuthology.orchestra.run.smithi007.stdout: "op_send_bytes": 21, 2024-03-23T13:21:32.610 INFO:teuthology.orchestra.run.smithi007.stdout: "op_resend": 3, 2024-03-23T13:21:32.610 INFO:teuthology.orchestra.run.smithi007.stdout: "op_reply": 583, 2024-03-23T13:21:32.610 INFO:teuthology.orchestra.run.smithi007.stdout: "op_latency": { 2024-03-23T13:21:32.610 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 583, 2024-03-23T13:21:32.610 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.455993176, 2024-03-23T13:21:32.611 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000782149 2024-03-23T13:21:32.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.611 INFO:teuthology.orchestra.run.smithi007.stdout: "op_inflight": 0, 2024-03-23T13:21:32.611 INFO:teuthology.orchestra.run.smithi007.stdout: "oplen_avg": { 2024-03-23T13:21:32.611 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 583, 2024-03-23T13:21:32.611 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 584 2024-03-23T13:21:32.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.611 INFO:teuthology.orchestra.run.smithi007.stdout: "op": 583, 2024-03-23T13:21:32.612 INFO:teuthology.orchestra.run.smithi007.stdout: "op_r": 583, 2024-03-23T13:21:32.612 INFO:teuthology.orchestra.run.smithi007.stdout: "op_w": 0, 2024-03-23T13:21:32.612 INFO:teuthology.orchestra.run.smithi007.stdout: "op_rmw": 0, 2024-03-23T13:21:32.612 INFO:teuthology.orchestra.run.smithi007.stdout: "op_pg": 0, 2024-03-23T13:21:32.612 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_stat": 335, 2024-03-23T13:21:32.612 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_create": 0, 2024-03-23T13:21:32.612 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_read": 246, 2024-03-23T13:21:32.612 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_write": 0, 2024-03-23T13:21:32.612 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_writefull": 0, 2024-03-23T13:21:32.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_writesame": 0, 2024-03-23T13:21:32.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_append": 0, 2024-03-23T13:21:32.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_zero": 0, 2024-03-23T13:21:32.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_truncate": 0, 2024-03-23T13:21:32.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_delete": 0, 2024-03-23T13:21:32.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_mapext": 0, 2024-03-23T13:21:32.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_sparse_read": 0, 2024-03-23T13:21:32.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_clonerange": 0, 2024-03-23T13:21:32.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_getxattr": 1, 2024-03-23T13:21:32.614 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_setxattr": 0, 2024-03-23T13:21:32.614 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_cmpxattr": 0, 2024-03-23T13:21:32.614 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_rmxattr": 0, 2024-03-23T13:21:32.614 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_resetxattrs": 0, 2024-03-23T13:21:32.614 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_call": 0, 2024-03-23T13:21:32.614 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_watch": 0, 2024-03-23T13:21:32.614 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_notify": 0, 2024-03-23T13:21:32.614 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T13:21:32.615 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_pgls": 0, 2024-03-23T13:21:32.615 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_pgls_filter": 0, 2024-03-23T13:21:32.615 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_other": 0, 2024-03-23T13:21:32.615 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_active": 0, 2024-03-23T13:21:32.615 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_send": 0, 2024-03-23T13:21:32.615 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_resend": 0, 2024-03-23T13:21:32.615 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_ping": 0, 2024-03-23T13:21:32.615 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_active": 0, 2024-03-23T13:21:32.616 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_send": 0, 2024-03-23T13:21:32.616 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_resend": 0, 2024-03-23T13:21:32.616 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_active": 0, 2024-03-23T13:21:32.616 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_send": 0, 2024-03-23T13:21:32.616 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_resend": 0, 2024-03-23T13:21:32.616 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_active": 0, 2024-03-23T13:21:32.616 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_send": 0, 2024-03-23T13:21:32.616 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_resend": 0, 2024-03-23T13:21:32.616 INFO:teuthology.orchestra.run.smithi007.stdout: "command_active": 0, 2024-03-23T13:21:32.617 INFO:teuthology.orchestra.run.smithi007.stdout: "command_send": 0, 2024-03-23T13:21:32.617 INFO:teuthology.orchestra.run.smithi007.stdout: "command_resend": 0, 2024-03-23T13:21:32.617 INFO:teuthology.orchestra.run.smithi007.stdout: "map_epoch": 87, 2024-03-23T13:21:32.617 INFO:teuthology.orchestra.run.smithi007.stdout: "map_full": 0, 2024-03-23T13:21:32.617 INFO:teuthology.orchestra.run.smithi007.stdout: "map_inc": 2, 2024-03-23T13:21:32.617 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_sessions": 5, 2024-03-23T13:21:32.617 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_session_open": 5, 2024-03-23T13:21:32.617 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_session_close": 0, 2024-03-23T13:21:32.618 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_laggy": 0, 2024-03-23T13:21:32.618 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_wr": 0, 2024-03-23T13:21:32.618 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_rd": 2, 2024-03-23T13:21:32.618 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_del": 0 2024-03-23T13:21:32.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.618 INFO:teuthology.orchestra.run.smithi007.stdout: "oft": { 2024-03-23T13:21:32.618 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_objs": 0, 2024-03-23T13:21:32.618 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_kv_pairs": 0, 2024-03-23T13:21:32.619 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_updates": 0, 2024-03-23T13:21:32.619 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_removes": 0 2024-03-23T13:21:32.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.619 INFO:teuthology.orchestra.run.smithi007.stdout: "osd-slow-ops": { 2024-03-23T13:21:32.619 INFO:teuthology.orchestra.run.smithi007.stdout: "slow_ops_count": 0 2024-03-23T13:21:32.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.619 INFO:teuthology.orchestra.run.smithi007.stdout: "purge_queue": { 2024-03-23T13:21:32.619 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_ops": 0, 2024-03-23T13:21:32.620 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T13:21:32.620 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing": 0, 2024-03-23T13:21:32.620 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_high_water": 0, 2024-03-23T13:21:32.620 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executed_ops": 0, 2024-03-23T13:21:32.620 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executed": 0, 2024-03-23T13:21:32.620 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_item_in_journal": 0 2024-03-23T13:21:32.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.620 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:21:32.620 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:32.621 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-23T13:21:32.621 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:32.621 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 666, 2024-03-23T13:21:32.621 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 4751116, 2024-03-23T13:21:32.621 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:32.621 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 666, 2024-03-23T13:21:32.621 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-23T13:21:32.621 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-23T13:21:32.622 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 666, 2024-03-23T13:21:32.622 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 4751116, 2024-03-23T13:21:32.622 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:32.622 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.622 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.622 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:32.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.623 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-objecter_bytes": { 2024-03-23T13:21:32.623 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:32.623 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-23T13:21:32.623 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:32.623 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-03-23T13:21:32.623 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-03-23T13:21:32.623 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:32.623 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 583, 2024-03-23T13:21:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 4171184, 2024-03-23T13:21:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 583, 2024-03-23T13:21:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 4171184, 2024-03-23T13:21:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:32.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.625 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-objecter_ops": { 2024-03-23T13:21:32.625 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:32.625 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 1024, 2024-03-23T13:21:32.625 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:32.625 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-03-23T13:21:32.625 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-03-23T13:21:32.626 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:32.626 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:32.626 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 583, 2024-03-23T13:21:32.626 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 583, 2024-03-23T13:21:32.626 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 583, 2024-03-23T13:21:32.626 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 583, 2024-03-23T13:21:32.626 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:32.626 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.627 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.627 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.627 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:32.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.627 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-write_buf_throttle": { 2024-03-23T13:21:32.627 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:32.627 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 3758096384, 2024-03-23T13:21:32.627 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:32.628 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-03-23T13:21:32.628 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-03-23T13:21:32.628 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:32.628 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:32.628 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-23T13:21:32.628 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-23T13:21:32.628 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 0, 2024-03-23T13:21:32.628 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 0, 2024-03-23T13:21:32.629 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:32.629 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.629 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.629 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:32.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:32.629 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-write_buf_throttle-0x55a7229c26e0": { 2024-03-23T13:21:32.629 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:32.629 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 3758096384, 2024-03-23T13:21:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-03-23T13:21:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-03-23T13:21:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-23T13:21:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-23T13:21:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 0, 2024-03-23T13:21:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 0, 2024-03-23T13:21:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:32.631 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:21:33.373 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:33 smithi007 ceph-mon[32184]: pgmap v402: 97 pgs: 97 active+clean; 43 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 79 KiB/s wr, 32 op/s 2024-03-23T13:21:33.400 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-23T13:21:33.400 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-23T13:21:33.401 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-03-23T13:21:33.401 DEBUG:tasks.check_counter:Getting stats from d 2024-03-23T13:21:33.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-23T13:21:33.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:33 smithi130 ceph-mon[36225]: pgmap v402: 97 pgs: 97 active+clean; 43 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 79 KiB/s wr, 32 op/s 2024-03-23T13:21:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:33 smithi155 ceph-mon[36157]: pgmap v402: 97 pgs: 97 active+clean; 43 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 79 KiB/s wr, 32 op/s 2024-03-23T13:21:33.647 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:21:35.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:35 smithi007 ceph-mon[32184]: pgmap v403: 97 pgs: 97 active+clean; 45 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 191 KiB/s wr, 38 op/s 2024-03-23T13:21:35.415 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:21:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:21:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 77, 2024-03-23T13:21:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 74, 2024-03-23T13:21:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 434240, 2024-03-23T13:21:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 44336, 2024-03-23T13:21:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-23T13:21:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-03-23T13:21:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.014555873, 2024-03-23T13:21:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.004034330, 2024-03-23T13:21:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.012204052, 2024-03-23T13:21:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.001301609, 2024-03-23T13:21:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 73, 2024-03-23T13:21:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.003237189, 2024-03-23T13:21:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000044345 2024-03-23T13:21:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 434240, 2024-03-23T13:21:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 44336 2024-03-23T13:21:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:21:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 1, 2024-03-23T13:21:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 56, 2024-03-23T13:21:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 160, 2024-03-23T13:21:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 87408, 2024-03-23T13:21:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 3, 2024-03-23T13:21:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-23T13:21:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.004269671, 2024-03-23T13:21:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.002632089, 2024-03-23T13:21:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001154836, 2024-03-23T13:21:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T13:21:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 54, 2024-03-23T13:21:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.002165785, 2024-03-23T13:21:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000040107 2024-03-23T13:21:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T13:21:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 87408 2024-03-23T13:21:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:21:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 4, 2024-03-23T13:21:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 4, 2024-03-23T13:21:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 2992, 2024-03-23T13:21:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 448, 2024-03-23T13:21:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-03-23T13:21:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-03-23T13:21:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.000627614, 2024-03-23T13:21:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000034639, 2024-03-23T13:21:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000570037, 2024-03-23T13:21:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T13:21:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 3, 2024-03-23T13:21:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000725928, 2024-03-23T13:21:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000241976 2024-03-23T13:21:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-23T13:21:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 448 2024-03-23T13:21:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-23T13:21:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-03-23T13:21:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-23T13:21:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-23T13:21:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:21:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-23T13:21:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:21:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:21:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:21:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:21:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:21:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:21:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:21:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:21:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:21:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:21:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:21:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:21:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:21:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-23T13:21:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:21:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-23T13:21:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:21:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:21:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:21:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:21:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:21:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:21:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:21:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-23T13:21:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:21:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:21:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:21:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-23T13:21:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-23T13:21:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-23T13:21:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:21:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:21:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:21:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:21:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148863, 2024-03-23T13:21:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 28, 2024-03-23T13:21:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-23T13:21:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-23T13:21:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-23T13:21:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-23T13:21:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:21:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-23T13:21:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:21:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-23T13:21:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-03-23T13:21:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-03-23T13:21:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:21:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:21:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-23T13:21:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-23T13:21:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-03-23T13:21:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-03-23T13:21:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-23T13:21:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-23T13:21:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-23T13:21:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-23T13:21:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:21:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-23T13:21:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 78, 2024-03-23T13:21:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 423388, 2024-03-23T13:21:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 78, 2024-03-23T13:21:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-23T13:21:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-23T13:21:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 78, 2024-03-23T13:21:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 423388, 2024-03-23T13:21:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:35.435 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:21:35.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:35 smithi130 ceph-mon[36225]: pgmap v403: 97 pgs: 97 active+clean; 45 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 191 KiB/s wr, 38 op/s 2024-03-23T13:21:35.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:35 smithi155 ceph-mon[36157]: pgmap v403: 97 pgs: 97 active+clean; 45 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 191 KiB/s wr, 38 op/s 2024-03-23T13:21:36.146 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-23T13:21:36.147 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-23T13:21:36.147 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-03-23T13:21:36.147 DEBUG:tasks.check_counter:Getting stats from g 2024-03-23T13:21:36.147 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-23T13:21:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:21:37.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:37 smithi130 ceph-mon[36225]: pgmap v404: 97 pgs: 97 active+clean; 46 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 317 KiB/s wr, 53 op/s 2024-03-23T13:21:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:37 smithi007 ceph-mon[32184]: pgmap v404: 97 pgs: 97 active+clean; 46 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 317 KiB/s wr, 53 op/s 2024-03-23T13:21:37.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:37 smithi155 ceph-mon[36157]: pgmap v404: 97 pgs: 97 active+clean; 46 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 317 KiB/s wr, 53 op/s 2024-03-23T13:21:38.056 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:21:38.056 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:21:38.056 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 235, 2024-03-23T13:21:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 235, 2024-03-23T13:21:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 80553, 2024-03-23T13:21:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 72290, 2024-03-23T13:21:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 6, 2024-03-23T13:21:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-23T13:21:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.024147489, 2024-03-23T13:21:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.009523109, 2024-03-23T13:21:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.005813096, 2024-03-23T13:21:38.058 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.005853046, 2024-03-23T13:21:38.058 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:38.058 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 234, 2024-03-23T13:21:38.058 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.011466453, 2024-03-23T13:21:38.058 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000049001 2024-03-23T13:21:38.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.058 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:38.059 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.059 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.059 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.059 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-23T13:21:38.059 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 448 2024-03-23T13:21:38.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.059 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:21:38.059 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 380, 2024-03-23T13:21:38.060 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 373, 2024-03-23T13:21:38.060 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 4134161, 2024-03-23T13:21:38.060 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 135073, 2024-03-23T13:21:38.060 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-23T13:21:38.060 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 3, 2024-03-23T13:21:38.060 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.033938567, 2024-03-23T13:21:38.060 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.009752641, 2024-03-23T13:21:38.060 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.018436245, 2024-03-23T13:21:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.007155481, 2024-03-23T13:21:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 372, 2024-03-23T13:21:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.011381142, 2024-03-23T13:21:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000030594 2024-03-23T13:21:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 476576, 2024-03-23T13:21:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 42960 2024-03-23T13:21:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:21:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 4, 2024-03-23T13:21:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 56, 2024-03-23T13:21:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1508, 2024-03-23T13:21:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 171440, 2024-03-23T13:21:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 5, 2024-03-23T13:21:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 4, 2024-03-23T13:21:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.005355440, 2024-03-23T13:21:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.002968609, 2024-03-23T13:21:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001765584, 2024-03-23T13:21:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000082318, 2024-03-23T13:21:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 54, 2024-03-23T13:21:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.003775649, 2024-03-23T13:21:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000069919 2024-03-23T13:21:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T13:21:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 170496 2024-03-23T13:21:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-23T13:21:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 1, 2024-03-23T13:21:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-23T13:21:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "finisher-MDSRank": { 2024-03-23T13:21:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "queue_len": 0, 2024-03-23T13:21:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "complete_latency": { 2024-03-23T13:21:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 1007, 2024-03-23T13:21:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.022380418, 2024-03-23T13:21:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000022224 2024-03-23T13:21:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "finisher-PurgeQueue": { 2024-03-23T13:21:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "queue_len": 0, 2024-03-23T13:21:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "complete_latency": { 2024-03-23T13:21:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-23T13:21:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "request": 0, 2024-03-23T13:21:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "reply": 0, 2024-03-23T13:21:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "reply_latency": { 2024-03-23T13:21:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "slow_reply": 0, 2024-03-23T13:21:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "forward": 0, 2024-03-23T13:21:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_fetch_complete": 0, 2024-03-23T13:21:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_fetch_keys": 0, 2024-03-23T13:21:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_commit": 0, 2024-03-23T13:21:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_split": 0, 2024-03-23T13:21:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_merge": 0, 2024-03-23T13:21:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes": 1448, 2024-03-23T13:21:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_top": 97, 2024-03-23T13:21:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_bottom": 1351, 2024-03-23T13:21:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_pin_tail": 0, 2024-03-23T13:21:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_pinned": 1313, 2024-03-23T13:21:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_expired": 0, 2024-03-23T13:21:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_with_caps": 0, 2024-03-23T13:21:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": 0, 2024-03-23T13:21:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "subtrees": 59, 2024-03-23T13:21:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse": 0, 2024-03-23T13:21:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_hit": 0, 2024-03-23T13:21:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_forward": 0, 2024-03-23T13:21:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_discover": 0, 2024-03-23T13:21:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_dir_fetch": 0, 2024-03-23T13:21:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_remote_ino": 0, 2024-03-23T13:21:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_lock": 0, 2024-03-23T13:21:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "load_cent": 0, 2024-03-23T13:21:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "q": 0, 2024-03-23T13:21:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "exported": 0, 2024-03-23T13:21:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "exported_inodes": 0, 2024-03-23T13:21:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "imported": 0, 2024-03-23T13:21:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "imported_inodes": 0, 2024-03-23T13:21:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_dir_fetch": 0, 2024-03-23T13:21:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_backtrace_fetch": 0, 2024-03-23T13:21:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_peer_discover": 0, 2024-03-23T13:21:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rfiles": 0, 2024-03-23T13:21:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rbytes": 0, 2024-03-23T13:21:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rsnaps": 0, 2024-03-23T13:21:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T13:21:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_set_tag": 0, 2024-03-23T13:21:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T13:21:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_inotable_repaired": 0, 2024-03-23T13:21:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dir_inodes": 0, 2024-03-23T13:21:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T13:21:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T13:21:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_file_inodes": 0, 2024-03-23T13:21:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_inode_file_caps": 0, 2024-03-23T13:21:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T13:21:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_grant": 0, 2024-03-23T13:21:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T13:21:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T13:21:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T13:21:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_caps": 0, 2024-03-23T13:21:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_caps_dirty": 0, 2024-03-23T13:21:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_cap_release": 0, 2024-03-23T13:21:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "process_request_cap_release": 0 2024-03-23T13:21:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache": { 2024-03-23T13:21:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_update": 0, 2024-03-23T13:21:38.092 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_update_receipt": 0, 2024-03-23T13:21:38.092 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_try_discover": 0, 2024-03-23T13:21:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_send_discover": 0, 2024-03-23T13:21:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_handle_discover": 0, 2024-03-23T13:21:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays": 0, 2024-03-23T13:21:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays_delayed": 0, 2024-03-23T13:21:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays_enqueuing": 0, 2024-03-23T13:21:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_created": 0, 2024-03-23T13:21:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_enqueued": 0, 2024-03-23T13:21:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_reintegrated": 0, 2024-03-23T13:21:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_migrated": 0, 2024-03-23T13:21:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_processing": 0, 2024-03-23T13:21:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_enqueued": 0, 2024-03-23T13:21:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_prioritized": 0, 2024-03-23T13:21:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_started": 0, 2024-03-23T13:21:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_completed": 0, 2024-03-23T13:21:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_quiesce_path": 0, 2024-03-23T13:21:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_quiesce_inode": 0, 2024-03-23T13:21:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T13:21:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_exportdir": 0, 2024-03-23T13:21:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_flush": 0, 2024-03-23T13:21:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_fragmentdir": 0, 2024-03-23T13:21:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_fragstats": 0, 2024-03-23T13:21:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_inodestats": 0 2024-03-23T13:21:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log": { 2024-03-23T13:21:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "evlrg": 0, 2024-03-23T13:21:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "evadd": 0, 2024-03-23T13:21:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "evex": 0, 2024-03-23T13:21:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "evtrm": 0, 2024-03-23T13:21:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "ev": 2241, 2024-03-23T13:21:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "evexg": 0, 2024-03-23T13:21:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "evexd": 0, 2024-03-23T13:21:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "segadd": 0, 2024-03-23T13:21:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "segex": 0, 2024-03-23T13:21:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "segtrm": 0, 2024-03-23T13:21:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "seg": 4, 2024-03-23T13:21:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "segmjr": 2, 2024-03-23T13:21:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "segexg": 0, 2024-03-23T13:21:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "segexd": 0, 2024-03-23T13:21:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "expos": 4194304, 2024-03-23T13:21:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "wrpos": 7775329, 2024-03-23T13:21:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "rdpos": 7774204, 2024-03-23T13:21:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "jlat": { 2024-03-23T13:21:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "replayed": 2241 2024-03-23T13:21:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_mem": { 2024-03-23T13:21:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 872, 2024-03-23T13:21:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "ino+": 1265, 2024-03-23T13:21:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "ino-": 393, 2024-03-23T13:21:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "dir": 77, 2024-03-23T13:21:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "dir+": 77, 2024-03-23T13:21:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "dir-": 0, 2024-03-23T13:21:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "dn": 1448, 2024-03-23T13:21:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "dn+": 1448, 2024-03-23T13:21:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "dn-": 0, 2024-03-23T13:21:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "cap": 0, 2024-03-23T13:21:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "cap+": 0, 2024-03-23T13:21:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "cap-": 0, 2024-03-23T13:21:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "rss": 53488, 2024-03-23T13:21:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "heap": 215420 2024-03-23T13:21:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server": { 2024-03-23T13:21:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "dispatch_client_request": 0, 2024-03-23T13:21:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "dispatch_server_request": 0, 2024-03-23T13:21:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_request": 0, 2024-03-23T13:21:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_session": 0, 2024-03-23T13:21:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_peer_request": 0, 2024-03-23T13:21:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "req_create_latency": { 2024-03-23T13:21:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getattr_latency": { 2024-03-23T13:21:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getfilelock_latency": { 2024-03-23T13:21:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "req_link_latency": { 2024-03-23T13:21:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookup_latency": { 2024-03-23T13:21:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookuphash_latency": { 2024-03-23T13:21:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupino_latency": { 2024-03-23T13:21:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupname_latency": { 2024-03-23T13:21:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupparent_latency": { 2024-03-23T13:21:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupsnap_latency": { 2024-03-23T13:21:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lssnap_latency": { 2024-03-23T13:21:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mkdir_latency": { 2024-03-23T13:21:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mknod_latency": { 2024-03-23T13:21:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mksnap_latency": { 2024-03-23T13:21:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "req_open_latency": { 2024-03-23T13:21:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "req_readdir_latency": { 2024-03-23T13:21:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rename_latency": { 2024-03-23T13:21:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "req_renamesnap_latency": { 2024-03-23T13:21:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "req_snapdiff_latency": { 2024-03-23T13:21:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmdir_latency": { 2024-03-23T13:21:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmsnap_latency": { 2024-03-23T13:21:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmxattr_latency": { 2024-03-23T13:21:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setattr_latency": { 2024-03-23T13:21:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setdirlayout_latency": { 2024-03-23T13:21:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setfilelock_latency": { 2024-03-23T13:21:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setlayout_latency": { 2024-03-23T13:21:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setxattr_latency": { 2024-03-23T13:21:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "req_symlink_latency": { 2024-03-23T13:21:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "req_unlink_latency": { 2024-03-23T13:21:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_revoke_eviction": 0, 2024-03-23T13:21:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_acquisition_throttle": 0, 2024-03-23T13:21:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getvxattr_latency": { 2024-03-23T13:21:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sessions": { 2024-03-23T13:21:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "session_count": 1, 2024-03-23T13:21:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "session_add": 1, 2024-03-23T13:21:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "session_remove": 0, 2024-03-23T13:21:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions_open": 1, 2024-03-23T13:21:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions_stale": 0, 2024-03-23T13:21:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "total_load": 0, 2024-03-23T13:21:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "average_load": 0, 2024-03-23T13:21:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "avg_session_uptime": 1214, 2024-03-23T13:21:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "mdthresh_evicted": 0 2024-03-23T13:21:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-23T13:21:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:21:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-23T13:21:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:21:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:21:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:21:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:21:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:21:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:21:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:21:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:21:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:21:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:21:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:21:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:21:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:21:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-23T13:21:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:21:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-23T13:21:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:21:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:21:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:21:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:21:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:21:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:21:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:21:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-23T13:21:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:21:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:21:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:21:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-23T13:21:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-23T13:21:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-23T13:21:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:21:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:21:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:21:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:21:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 229290, 2024-03-23T13:21:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 916, 2024-03-23T13:21:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-23T13:21:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-23T13:21:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-23T13:21:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-23T13:21:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:21:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-23T13:21:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:21:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-23T13:21:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 7528, 2024-03-23T13:21:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 158, 2024-03-23T13:21:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:21:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:21:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-23T13:21:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-23T13:21:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 3102482, 2024-03-23T13:21:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 26247, 2024-03-23T13:21:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-23T13:21:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-23T13:21:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-23T13:21:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-23T13:21:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter": { 2024-03-23T13:21:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "op_active": 0, 2024-03-23T13:21:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "op_laggy": 0, 2024-03-23T13:21:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "op_send": 530, 2024-03-23T13:21:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "op_send_bytes": 21, 2024-03-23T13:21:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "op_resend": 3, 2024-03-23T13:21:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "op_reply": 530, 2024-03-23T13:21:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "op_latency": { 2024-03-23T13:21:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 530, 2024-03-23T13:21:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.306995405, 2024-03-23T13:21:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000579236 2024-03-23T13:21:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "op_inflight": 0, 2024-03-23T13:21:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "oplen_avg": { 2024-03-23T13:21:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 530, 2024-03-23T13:21:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 531 2024-03-23T13:21:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "op": 530, 2024-03-23T13:21:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "op_r": 530, 2024-03-23T13:21:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "op_w": 0, 2024-03-23T13:21:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "op_rmw": 0, 2024-03-23T13:21:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "op_pg": 0, 2024-03-23T13:21:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_stat": 274, 2024-03-23T13:21:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_create": 0, 2024-03-23T13:21:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_read": 254, 2024-03-23T13:21:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_write": 0, 2024-03-23T13:21:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_writefull": 0, 2024-03-23T13:21:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_writesame": 0, 2024-03-23T13:21:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_append": 0, 2024-03-23T13:21:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_zero": 0, 2024-03-23T13:21:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_truncate": 0, 2024-03-23T13:21:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_delete": 0, 2024-03-23T13:21:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_mapext": 0, 2024-03-23T13:21:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_sparse_read": 0, 2024-03-23T13:21:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_clonerange": 0, 2024-03-23T13:21:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_getxattr": 1, 2024-03-23T13:21:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_setxattr": 0, 2024-03-23T13:21:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_cmpxattr": 0, 2024-03-23T13:21:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_rmxattr": 0, 2024-03-23T13:21:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_resetxattrs": 0, 2024-03-23T13:21:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_call": 0, 2024-03-23T13:21:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_watch": 0, 2024-03-23T13:21:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_notify": 0, 2024-03-23T13:21:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T13:21:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_pgls": 0, 2024-03-23T13:21:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_pgls_filter": 0, 2024-03-23T13:21:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_other": 0, 2024-03-23T13:21:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_active": 0, 2024-03-23T13:21:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_send": 0, 2024-03-23T13:21:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_resend": 0, 2024-03-23T13:21:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_ping": 0, 2024-03-23T13:21:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_active": 0, 2024-03-23T13:21:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_send": 0, 2024-03-23T13:21:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_resend": 0, 2024-03-23T13:21:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_active": 0, 2024-03-23T13:21:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_send": 0, 2024-03-23T13:21:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_resend": 0, 2024-03-23T13:21:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_active": 0, 2024-03-23T13:21:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_send": 0, 2024-03-23T13:21:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_resend": 0, 2024-03-23T13:21:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "command_active": 0, 2024-03-23T13:21:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "command_send": 0, 2024-03-23T13:21:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "command_resend": 0, 2024-03-23T13:21:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "map_epoch": 87, 2024-03-23T13:21:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "map_full": 0, 2024-03-23T13:21:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "map_inc": 2, 2024-03-23T13:21:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_sessions": 6, 2024-03-23T13:21:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_session_open": 6, 2024-03-23T13:21:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_session_close": 0, 2024-03-23T13:21:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_laggy": 0, 2024-03-23T13:21:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_wr": 0, 2024-03-23T13:21:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_rd": 2, 2024-03-23T13:21:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_del": 0 2024-03-23T13:21:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "oft": { 2024-03-23T13:21:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_objs": 0, 2024-03-23T13:21:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_kv_pairs": 0, 2024-03-23T13:21:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_updates": 0, 2024-03-23T13:21:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_removes": 0 2024-03-23T13:21:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd-slow-ops": { 2024-03-23T13:21:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "slow_ops_count": 0 2024-03-23T13:21:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "purge_queue": { 2024-03-23T13:21:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_ops": 0, 2024-03-23T13:21:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T13:21:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing": 0, 2024-03-23T13:21:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_high_water": 0, 2024-03-23T13:21:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executed_ops": 0, 2024-03-23T13:21:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executed": 0, 2024-03-23T13:21:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_item_in_journal": 0 2024-03-23T13:21:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:21:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-23T13:21:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 615, 2024-03-23T13:21:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 4153590, 2024-03-23T13:21:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 615, 2024-03-23T13:21:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-23T13:21:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-23T13:21:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 615, 2024-03-23T13:21:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 4153590, 2024-03-23T13:21:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-objecter_bytes": { 2024-03-23T13:21:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-23T13:21:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-03-23T13:21:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-03-23T13:21:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 530, 2024-03-23T13:21:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 3581030, 2024-03-23T13:21:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 530, 2024-03-23T13:21:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 3581030, 2024-03-23T13:21:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-objecter_ops": { 2024-03-23T13:21:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 1024, 2024-03-23T13:21:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-03-23T13:21:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-03-23T13:21:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 530, 2024-03-23T13:21:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 530, 2024-03-23T13:21:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 530, 2024-03-23T13:21:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 530, 2024-03-23T13:21:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:38.147 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.147 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.147 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:38.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.147 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-write_buf_throttle": { 2024-03-23T13:21:38.147 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:38.147 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 3758096384, 2024-03-23T13:21:38.147 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:38.148 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-03-23T13:21:38.148 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-03-23T13:21:38.148 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:38.148 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:38.148 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-23T13:21:38.148 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-23T13:21:38.148 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 0, 2024-03-23T13:21:38.148 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 0, 2024-03-23T13:21:38.148 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:38.149 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.149 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.149 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:38.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:38.149 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-write_buf_throttle-0x55d82d9ba6e0": { 2024-03-23T13:21:38.150 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:38.150 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 3758096384, 2024-03-23T13:21:38.150 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:38.150 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-03-23T13:21:38.150 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-03-23T13:21:38.150 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:38.150 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:38.150 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-23T13:21:38.151 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-23T13:21:38.151 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 0, 2024-03-23T13:21:38.151 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 0, 2024-03-23T13:21:38.151 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:38.151 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:38.151 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:38.151 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:38.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:38.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:38.152 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:21:38.821 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-23T13:21:38.821 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-23T13:21:38.821 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-03-23T13:21:38.821 DEBUG:tasks.check_counter:Getting stats from j 2024-03-23T13:21:38.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-23T13:21:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T13:21:39.445 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:39 smithi007 ceph-mon[32184]: pgmap v405: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 333 KiB/s wr, 60 op/s 2024-03-23T13:21:39.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:39 smithi130 ceph-mon[36225]: pgmap v405: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 333 KiB/s wr, 60 op/s 2024-03-23T13:21:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:39 smithi155 ceph-mon[36157]: pgmap v405: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 333 KiB/s wr, 60 op/s 2024-03-23T13:21:40.806 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-23T13:21:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:21:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-03-23T13:21:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-03-23T13:21:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-03-23T13:21:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-03-23T13:21:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-23T13:21:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-03-23T13:21:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.001135374, 2024-03-23T13:21:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T13:21:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000279790, 2024-03-23T13:21:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T13:21:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T13:21:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T13:21:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:21:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 83, 2024-03-23T13:21:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 78, 2024-03-23T13:21:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 437776, 2024-03-23T13:21:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 46080, 2024-03-23T13:21:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-03-23T13:21:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-03-23T13:21:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.013059524, 2024-03-23T13:21:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.004241674, 2024-03-23T13:21:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.009815263, 2024-03-23T13:21:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.001355585, 2024-03-23T13:21:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 78, 2024-03-23T13:21:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.004173372, 2024-03-23T13:21:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000053504 2024-03-23T13:21:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 437776, 2024-03-23T13:21:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 46080 2024-03-23T13:21:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:21:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 1, 2024-03-23T13:21:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 57, 2024-03-23T13:21:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 160, 2024-03-23T13:21:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 87584, 2024-03-23T13:21:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 3, 2024-03-23T13:21:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-23T13:21:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.004728895, 2024-03-23T13:21:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.003234129, 2024-03-23T13:21:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000975488, 2024-03-23T13:21:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T13:21:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 55, 2024-03-23T13:21:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.002562725, 2024-03-23T13:21:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000046595 2024-03-23T13:21:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T13:21:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 87584 2024-03-23T13:21:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-23T13:21:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-03-23T13:21:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-23T13:21:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-23T13:21:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:21:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-23T13:21:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:21:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:21:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:21:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:21:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:21:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:21:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:21:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:21:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:21:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:21:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:21:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:21:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:21:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-23T13:21:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:21:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-23T13:21:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:21:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:21:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:21:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:21:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:21:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:21:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:21:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-23T13:21:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:21:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:21:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:21:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-23T13:21:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-23T13:21:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-23T13:21:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:21:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:21:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:21:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:21:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148903, 2024-03-23T13:21:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 29, 2024-03-23T13:21:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-23T13:21:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-23T13:21:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-23T13:21:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-23T13:21:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:21:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-23T13:21:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:21:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-23T13:21:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-03-23T13:21:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-03-23T13:21:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:21:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:21:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-23T13:21:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-23T13:21:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-03-23T13:21:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-03-23T13:21:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-23T13:21:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-23T13:21:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-23T13:21:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-23T13:21:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-23T13:21:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:21:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-23T13:21:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-23T13:21:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-23T13:21:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 80, 2024-03-23T13:21:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 423648, 2024-03-23T13:21:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 80, 2024-03-23T13:21:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-23T13:21:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-23T13:21:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 80, 2024-03-23T13:21:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 423648, 2024-03-23T13:21:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-23T13:21:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-23T13:21:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-23T13:21:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-23T13:21:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-23T13:21:40.825 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-23T13:21:41.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:41 smithi130 ceph-mon[36225]: pgmap v406: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 313 KiB/s wr, 49 op/s 2024-03-23T13:21:41.450 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:41 smithi007 ceph-mon[32184]: pgmap v406: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 313 KiB/s wr, 49 op/s 2024-03-23T13:21:41.479 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-23T13:21:41.479 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-23T13:21:41.480 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-03-23T13:21:41.480 DEBUG:tasks.check_counter:Getting stats from b 2024-03-23T13:21:41.480 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-23T13:21:41.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:41 smithi155 ceph-mon[36157]: pgmap v406: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 313 KiB/s wr, 49 op/s 2024-03-23T13:21:41.733 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:21:43.386 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:43 smithi130 ceph-mon[36225]: pgmap v407: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 314 KiB/s wr, 52 op/s 2024-03-23T13:21:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:43 smithi155 ceph-mon[36157]: pgmap v407: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 314 KiB/s wr, 52 op/s 2024-03-23T13:21:43.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:43 smithi007 ceph-mon[32184]: pgmap v407: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 314 KiB/s wr, 52 op/s 2024-03-23T13:21:43.651 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-03-23T13:21:43.652 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:21:43.652 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 11410, 2024-03-23T13:21:43.652 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 6884, 2024-03-23T13:21:43.652 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 3865029, 2024-03-23T13:21:43.652 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 17668819, 2024-03-23T13:21:43.652 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 12, 2024-03-23T13:21:43.652 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-03-23T13:21:43.652 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.465961726, 2024-03-23T13:21:43.653 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.190255253, 2024-03-23T13:21:43.653 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.254439801, 2024-03-23T13:21:43.653 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.072946260, 2024-03-23T13:21:43.653 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:43.653 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 6882, 2024-03-23T13:21:43.653 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 4.516732490, 2024-03-23T13:21:43.653 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000656311 2024-03-23T13:21:43.653 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.653 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:43.654 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 13099, 2024-03-23T13:21:43.654 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.001240913, 2024-03-23T13:21:43.654 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000094 2024-03-23T13:21:43.654 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.654 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T13:21:43.654 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T13:21:43.654 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.654 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:21:43.654 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 18150, 2024-03-23T13:21:43.655 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 35559, 2024-03-23T13:21:43.655 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 6261109, 2024-03-23T13:21:43.655 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 28845965, 2024-03-23T13:21:43.655 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 13, 2024-03-23T13:21:43.655 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 7, 2024-03-23T13:21:43.655 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 1.186012978, 2024-03-23T13:21:43.655 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.530463658, 2024-03-23T13:21:43.655 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.913609735, 2024-03-23T13:21:43.656 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.118294353, 2024-03-23T13:21:43.656 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:43.656 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 35558, 2024-03-23T13:21:43.656 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 6.436740441, 2024-03-23T13:21:43.656 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000181020 2024-03-23T13:21:43.656 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.656 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:43.656 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 43632, 2024-03-23T13:21:43.656 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.007550144, 2024-03-23T13:21:43.657 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000173 2024-03-23T13:21:43.657 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.657 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 563376, 2024-03-23T13:21:43.657 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 79504 2024-03-23T13:21:43.657 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.657 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:21:43.657 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 7345, 2024-03-23T13:21:43.657 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 4828, 2024-03-23T13:21:43.657 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 2294447, 2024-03-23T13:21:43.658 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 7273604, 2024-03-23T13:21:43.658 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 22, 2024-03-23T13:21:43.658 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 9, 2024-03-23T13:21:43.658 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.407381229, 2024-03-23T13:21:43.658 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.193176509, 2024-03-23T13:21:43.658 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.224763332, 2024-03-23T13:21:43.658 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.045558795, 2024-03-23T13:21:43.658 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:43.658 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 4827, 2024-03-23T13:21:43.659 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 1.294231589, 2024-03-23T13:21:43.659 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000268123 2024-03-23T13:21:43.659 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.659 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:43.659 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 9055, 2024-03-23T13:21:43.659 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.001103646, 2024-03-23T13:21:43.659 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000121 2024-03-23T13:21:43.659 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.659 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-03-23T13:21:43.660 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 176560 2024-03-23T13:21:43.660 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.660 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-03-23T13:21:43.660 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-03-23T13:21:43.660 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-03-23T13:21:43.660 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.660 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-03-23T13:21:43.660 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-03-23T13:21:43.660 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-03-23T13:21:43.660 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1323, 2024-03-23T13:21:43.661 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 9.794075928, 2024-03-23T13:21:43.661 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.007402929 2024-03-23T13:21:43.661 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:43.661 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.661 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-03-23T13:21:43.661 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-03-23T13:21:43.661 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-03-23T13:21:43.661 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2665, 2024-03-23T13:21:43.661 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.716672137, 2024-03-23T13:21:43.662 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000268920 2024-03-23T13:21:43.662 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:43.662 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.662 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-03-23T13:21:43.662 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 9099, 2024-03-23T13:21:43.662 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 8874, 2024-03-23T13:21:43.662 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-03-23T13:21:43.662 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 8874, 2024-03-23T13:21:43.662 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 31.454685929, 2024-03-23T13:21:43.663 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.003544589 2024-03-23T13:21:43.663 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.663 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-03-23T13:21:43.663 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 225, 2024-03-23T13:21:43.663 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-03-23T13:21:43.663 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-03-23T13:21:43.663 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 479, 2024-03-23T13:21:43.663 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 22, 2024-03-23T13:21:43.663 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-03-23T13:21:43.664 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 919, 2024-03-23T13:21:43.664 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 484, 2024-03-23T13:21:43.664 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 393, 2024-03-23T13:21:43.664 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 42, 2024-03-23T13:21:43.664 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 227, 2024-03-23T13:21:43.664 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 4514, 2024-03-23T13:21:43.664 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 48, 2024-03-23T13:21:43.664 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 48, 2024-03-23T13:21:43.664 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 119, 2024-03-23T13:21:43.665 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 10615, 2024-03-23T13:21:43.665 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 9842, 2024-03-23T13:21:43.665 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 182, 2024-03-23T13:21:43.665 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 2, 2024-03-23T13:21:43.665 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-03-23T13:21:43.665 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-03-23T13:21:43.665 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 3, 2024-03-23T13:21:43.665 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 29214, 2024-03-23T13:21:43.665 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 21, 2024-03-23T13:21:43.666 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 187, 2024-03-23T13:21:43.666 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 4575, 2024-03-23T13:21:43.666 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-03-23T13:21:43.666 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-03-23T13:21:43.666 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-03-23T13:21:43.666 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-03-23T13:21:43.666 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-03-23T13:21:43.666 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 3452, 2024-03-23T13:21:43.667 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 26211650, 2024-03-23T13:21:43.667 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-03-23T13:21:43.667 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T13:21:43.667 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-03-23T13:21:43.667 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T13:21:43.667 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-03-23T13:21:43.667 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-03-23T13:21:43.667 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T13:21:43.667 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T13:21:43.668 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-03-23T13:21:43.668 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 18, 2024-03-23T13:21:43.668 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 169, 2024-03-23T13:21:43.668 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 7245, 2024-03-23T13:21:43.668 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 6, 2024-03-23T13:21:43.668 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T13:21:43.668 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T13:21:43.668 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 4512, 2024-03-23T13:21:43.668 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 3927, 2024-03-23T13:21:43.669 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 186, 2024-03-23T13:21:43.669 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 2237 2024-03-23T13:21:43.669 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.669 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-03-23T13:21:43.669 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-03-23T13:21:43.669 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-03-23T13:21:43.669 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-03-23T13:21:43.669 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 2, 2024-03-23T13:21:43.670 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 214, 2024-03-23T13:21:43.670 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 1, 2024-03-23T13:21:43.670 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-03-23T13:21:43.670 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-03-23T13:21:43.670 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 1541, 2024-03-23T13:21:43.670 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 1538, 2024-03-23T13:21:43.670 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 2, 2024-03-23T13:21:43.670 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-03-23T13:21:43.671 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-03-23T13:21:43.671 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-03-23T13:21:43.671 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-03-23T13:21:43.671 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-03-23T13:21:43.671 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-03-23T13:21:43.671 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-03-23T13:21:43.671 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-03-23T13:21:43.671 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T13:21:43.671 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 187, 2024-03-23T13:21:43.672 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-03-23T13:21:43.672 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 22, 2024-03-23T13:21:43.672 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-03-23T13:21:43.672 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-03-23T13:21:43.672 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.672 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-03-23T13:21:43.672 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-03-23T13:21:43.672 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 13700, 2024-03-23T13:21:43.672 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 13115, 2024-03-23T13:21:43.673 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 13115, 2024-03-23T13:21:43.673 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 585, 2024-03-23T13:21:43.673 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-03-23T13:21:43.673 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 3075, 2024-03-23T13:21:43.673 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 23, 2024-03-23T13:21:43.673 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 21, 2024-03-23T13:21:43.673 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 21, 2024-03-23T13:21:43.673 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 2, 2024-03-23T13:21:43.673 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 1, 2024-03-23T13:21:43.674 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-03-23T13:21:43.674 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 4, 2024-03-23T13:21:43.674 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 32712541, 2024-03-23T13:21:43.674 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 33997205, 2024-03-23T13:21:43.674 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 0, 2024-03-23T13:21:43.674 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-03-23T13:21:43.674 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 674, 2024-03-23T13:21:43.674 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 9.708913118, 2024-03-23T13:21:43.675 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.014404915 2024-03-23T13:21:43.675 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.675 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 0 2024-03-23T13:21:43.675 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.675 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-03-23T13:21:43.675 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 763, 2024-03-23T13:21:43.675 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 5833, 2024-03-23T13:21:43.675 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 5070, 2024-03-23T13:21:43.675 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 182, 2024-03-23T13:21:43.676 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 283, 2024-03-23T13:21:43.676 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 101, 2024-03-23T13:21:43.676 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 919, 2024-03-23T13:21:43.676 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 6935, 2024-03-23T13:21:43.676 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 6016, 2024-03-23T13:21:43.676 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 48, 2024-03-23T13:21:43.676 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 5512, 2024-03-23T13:21:43.676 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 5464, 2024-03-23T13:21:43.676 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 88788, 2024-03-23T13:21:43.677 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 231804 2024-03-23T13:21:43.677 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.677 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-03-23T13:21:43.677 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 9419, 2024-03-23T13:21:43.677 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 2962, 2024-03-23T13:21:43.677 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 9099, 2024-03-23T13:21:43.677 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 198, 2024-03-23T13:21:43.677 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 5884, 2024-03-23T13:21:43.678 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-03-23T13:21:43.678 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 5420, 2024-03-23T13:21:43.678 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 18.324937595, 2024-03-23T13:21:43.678 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.003380984 2024-03-23T13:21:43.678 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.678 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-03-23T13:21:43.678 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 132, 2024-03-23T13:21:43.678 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 1.052270553, 2024-03-23T13:21:43.679 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.007971746 2024-03-23T13:21:43.679 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.679 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-03-23T13:21:43.679 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.679 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.679 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.679 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.679 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-03-23T13:21:43.679 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2, 2024-03-23T13:21:43.680 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.003423743, 2024-03-23T13:21:43.680 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001711871 2024-03-23T13:21:43.680 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.680 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-03-23T13:21:43.680 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1227, 2024-03-23T13:21:43.680 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 1.101837285, 2024-03-23T13:21:43.680 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000897992 2024-03-23T13:21:43.680 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.681 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-03-23T13:21:43.681 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.681 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.681 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.681 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.681 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-03-23T13:21:43.681 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1, 2024-03-23T13:21:43.681 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000139434, 2024-03-23T13:21:43.681 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000139434 2024-03-23T13:21:43.682 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.682 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-03-23T13:21:43.682 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.682 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.682 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.682 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.682 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-03-23T13:21:43.682 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.683 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.683 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.683 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.683 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-03-23T13:21:43.683 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.683 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.683 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.683 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.684 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-03-23T13:21:43.684 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.684 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.685 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.685 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.685 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-03-23T13:21:43.685 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 82, 2024-03-23T13:21:43.685 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 3.887067744, 2024-03-23T13:21:43.685 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.047403265 2024-03-23T13:21:43.685 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.685 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-03-23T13:21:43.685 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.686 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.686 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.686 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.686 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-03-23T13:21:43.686 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.686 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.686 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.686 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.687 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-03-23T13:21:43.687 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 337, 2024-03-23T13:21:43.687 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.066319480, 2024-03-23T13:21:43.687 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000196793 2024-03-23T13:21:43.687 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.687 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-03-23T13:21:43.687 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 75, 2024-03-23T13:21:43.687 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.790493160, 2024-03-23T13:21:43.687 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.010539908 2024-03-23T13:21:43.688 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.688 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-03-23T13:21:43.688 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 66, 2024-03-23T13:21:43.688 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.195834372, 2024-03-23T13:21:43.688 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.002967187 2024-03-23T13:21:43.688 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.688 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-03-23T13:21:43.688 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.688 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.689 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.689 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.689 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-03-23T13:21:43.689 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.689 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.689 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.689 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.689 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-03-23T13:21:43.690 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 22, 2024-03-23T13:21:43.690 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.674225985, 2024-03-23T13:21:43.690 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.030646635 2024-03-23T13:21:43.690 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.690 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-03-23T13:21:43.690 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.690 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.691 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.691 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.691 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-03-23T13:21:43.691 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.691 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.691 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.691 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.691 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-03-23T13:21:43.691 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 7, 2024-03-23T13:21:43.692 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.226266232, 2024-03-23T13:21:43.692 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.032323747 2024-03-23T13:21:43.692 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.692 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-03-23T13:21:43.692 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.692 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.692 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.692 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.693 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-03-23T13:21:43.693 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 8, 2024-03-23T13:21:43.693 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.003508831, 2024-03-23T13:21:43.693 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000438603 2024-03-23T13:21:43.693 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.693 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-03-23T13:21:43.693 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.693 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.694 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.694 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.694 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-03-23T13:21:43.694 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 7, 2024-03-23T13:21:43.694 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.018040847, 2024-03-23T13:21:43.694 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.002577263 2024-03-23T13:21:43.694 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.694 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-03-23T13:21:43.695 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 6, 2024-03-23T13:21:43.695 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.005218943, 2024-03-23T13:21:43.695 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000869823 2024-03-23T13:21:43.695 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.695 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-03-23T13:21:43.695 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1482, 2024-03-23T13:21:43.695 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 5.105101725, 2024-03-23T13:21:43.695 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.003444738 2024-03-23T13:21:43.695 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.696 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-03-23T13:21:43.696 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-03-23T13:21:43.696 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-03-23T13:21:43.696 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.696 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.696 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.696 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:43.696 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.696 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-03-23T13:21:43.697 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 1, 2024-03-23T13:21:43.697 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 2, 2024-03-23T13:21:43.697 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 1, 2024-03-23T13:21:43.697 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 1, 2024-03-23T13:21:43.697 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-03-23T13:21:43.697 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 5831, 2024-03-23T13:21:43.697 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 5831, 2024-03-23T13:21:43.697 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 122, 2024-03-23T13:21:43.698 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-03-23T13:21:43.698 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.698 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-03-23T13:21:43.698 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 360, 2024-03-23T13:21:43.698 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 360, 2024-03-23T13:21:43.698 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:21:43.698 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:21:43.698 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:21:43.699 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:21:43.699 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:21:43.699 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:21:43.699 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:21:43.699 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:21:43.699 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:21:43.699 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:21:43.699 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:21:43.699 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:21:43.700 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:21:43.700 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-03-23T13:21:43.700 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:21:43.700 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-03-23T13:21:43.700 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:21:43.700 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:21:43.700 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:21:43.700 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:21:43.700 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:21:43.701 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:21:43.701 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:21:43.701 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-03-23T13:21:43.701 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:21:43.701 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:21:43.701 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:21:43.701 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-03-23T13:21:43.701 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-03-23T13:21:43.702 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-03-23T13:21:43.702 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:21:43.702 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:21:43.702 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:21:43.702 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:21:43.702 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 330973, 2024-03-23T13:21:43.702 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 855, 2024-03-23T13:21:43.702 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 264, 2024-03-23T13:21:43.703 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 3, 2024-03-23T13:21:43.703 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-03-23T13:21:43.703 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-03-23T13:21:43.703 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:21:43.703 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-03-23T13:21:43.703 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:21:43.703 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-03-23T13:21:43.703 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 7528, 2024-03-23T13:21:43.703 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 158, 2024-03-23T13:21:43.704 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:21:43.704 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:21:43.704 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-03-23T13:21:43.704 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-03-23T13:21:43.704 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 2758720, 2024-03-23T13:21:43.704 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 35544, 2024-03-23T13:21:43.704 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-03-23T13:21:43.704 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-03-23T13:21:43.705 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-03-23T13:21:43.705 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-03-23T13:21:43.705 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.705 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-03-23T13:21:43.705 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-03-23T13:21:43.705 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-03-23T13:21:43.705 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 7442, 2024-03-23T13:21:43.705 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 36094744, 2024-03-23T13:21:43.705 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 37, 2024-03-23T13:21:43.706 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 7442, 2024-03-23T13:21:43.706 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-03-23T13:21:43.706 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 7442, 2024-03-23T13:21:43.706 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 584.727998873, 2024-03-23T13:21:43.706 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.078571351 2024-03-23T13:21:43.706 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.706 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-03-23T13:21:43.706 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-03-23T13:21:43.707 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 7442, 2024-03-23T13:21:43.707 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 16226 2024-03-23T13:21:43.707 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.707 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 7442, 2024-03-23T13:21:43.707 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 43, 2024-03-23T13:21:43.707 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 7399, 2024-03-23T13:21:43.707 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-03-23T13:21:43.707 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-03-23T13:21:43.707 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 216, 2024-03-23T13:21:43.708 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 3982, 2024-03-23T13:21:43.708 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 43, 2024-03-23T13:21:43.708 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 717, 2024-03-23T13:21:43.708 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 60, 2024-03-23T13:21:43.708 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-03-23T13:21:43.708 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-03-23T13:21:43.708 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-03-23T13:21:43.708 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 6, 2024-03-23T13:21:43.709 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 2079, 2024-03-23T13:21:43.709 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-03-23T13:21:43.709 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-03-23T13:21:43.709 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-03-23T13:21:43.709 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 0, 2024-03-23T13:21:43.709 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 7964, 2024-03-23T13:21:43.710 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-03-23T13:21:43.710 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-03-23T13:21:43.710 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-03-23T13:21:43.710 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-03-23T13:21:43.710 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-03-23T13:21:43.710 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-03-23T13:21:43.710 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T13:21:43.711 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-03-23T13:21:43.711 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-03-23T13:21:43.711 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 6, 2024-03-23T13:21:43.711 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-03-23T13:21:43.711 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-03-23T13:21:43.711 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-03-23T13:21:43.711 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-03-23T13:21:43.712 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-03-23T13:21:43.712 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-03-23T13:21:43.712 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-03-23T13:21:43.712 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-03-23T13:21:43.712 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-03-23T13:21:43.712 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-03-23T13:21:43.712 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-03-23T13:21:43.712 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-03-23T13:21:43.713 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-03-23T13:21:43.713 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-03-23T13:21:43.713 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-03-23T13:21:43.713 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-03-23T13:21:43.713 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 87, 2024-03-23T13:21:43.713 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-03-23T13:21:43.713 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 3, 2024-03-23T13:21:43.713 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 12, 2024-03-23T13:21:43.713 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 12, 2024-03-23T13:21:43.714 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-03-23T13:21:43.714 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-03-23T13:21:43.714 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 1046, 2024-03-23T13:21:43.714 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 0, 2024-03-23T13:21:43.714 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 107 2024-03-23T13:21:43.714 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.714 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-03-23T13:21:43.714 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 1, 2024-03-23T13:21:43.715 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-03-23T13:21:43.715 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-03-23T13:21:43.715 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-03-23T13:21:43.715 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.715 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-03-23T13:21:43.715 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-03-23T13:21:43.715 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.715 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-03-23T13:21:43.715 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-03-23T13:21:43.716 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 6, 2024-03-23T13:21:43.716 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-03-23T13:21:43.716 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 6, 2024-03-23T13:21:43.716 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 1538, 2024-03-23T13:21:43.716 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 1538, 2024-03-23T13:21:43.716 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-03-23T13:21:43.716 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.716 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:21:43.716 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:43.717 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-03-23T13:21:43.717 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:43.717 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 36901, 2024-03-23T13:21:43.717 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 9083894, 2024-03-23T13:21:43.717 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:43.717 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 36901, 2024-03-23T13:21:43.717 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:43.717 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:43.718 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 36901, 2024-03-23T13:21:43.718 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 9083894, 2024-03-23T13:21:43.718 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:43.718 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.718 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.718 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.718 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:43.718 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.719 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-03-23T13:21:43.719 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:43.719 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-03-23T13:21:43.719 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:43.719 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:43.719 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:43.720 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:43.720 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:43.720 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 7442, 2024-03-23T13:21:43.720 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 36250082, 2024-03-23T13:21:43.720 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 7442, 2024-03-23T13:21:43.720 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 36250082, 2024-03-23T13:21:43.720 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:43.720 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.721 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.721 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.721 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:43.721 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.721 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-03-23T13:21:43.721 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:43.721 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-03-23T13:21:43.721 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:43.721 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:43.722 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:43.722 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:43.722 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:43.722 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 7442, 2024-03-23T13:21:43.722 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 7442, 2024-03-23T13:21:43.722 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 7442, 2024-03-23T13:21:43.722 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 7442, 2024-03-23T13:21:43.723 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:43.723 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.723 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.723 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.723 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:43.723 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.723 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-03-23T13:21:43.723 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:43.723 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-03-23T13:21:43.724 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:43.724 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 1538, 2024-03-23T13:21:43.724 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 155338, 2024-03-23T13:21:43.724 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:43.724 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 1538, 2024-03-23T13:21:43.724 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:43.724 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:43.724 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 43, 2024-03-23T13:21:43.724 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 155338, 2024-03-23T13:21:43.725 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:43.725 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.725 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.725 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.725 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:43.725 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:43.725 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x55bff3378500": { 2024-03-23T13:21:43.725 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:43.725 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-03-23T13:21:43.726 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:43.726 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 13700, 2024-03-23T13:21:43.726 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 29802901, 2024-03-23T13:21:43.726 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:43.726 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 13700, 2024-03-23T13:21:43.726 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:43.726 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:43.726 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 674, 2024-03-23T13:21:43.727 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 29802901, 2024-03-23T13:21:43.727 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:43.727 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:43.727 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:43.727 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:43.727 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:43.727 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:43.727 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-03-23T13:21:44.404 INFO:tasks.check_counter:Daemon mds.b mds.exported=187 2024-03-23T13:21:44.404 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-23T13:21:44.404 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-03-23T13:21:44.404 DEBUG:tasks.check_counter:Getting stats from e 2024-03-23T13:21:44.404 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-23T13:21:44.736 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:21:45.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:45 smithi007 ceph-mon[32184]: pgmap v408: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 291 KiB/s wr, 50 op/s 2024-03-23T13:21:45.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:45 smithi130 ceph-mon[36225]: pgmap v408: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 291 KiB/s wr, 50 op/s 2024-03-23T13:21:45.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:45 smithi155 ceph-mon[36157]: pgmap v408: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 291 KiB/s wr, 50 op/s 2024-03-23T13:21:46.560 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-03-23T13:21:46.561 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:21:46.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 454, 2024-03-23T13:21:46.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 456, 2024-03-23T13:21:46.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 8527770, 2024-03-23T13:21:46.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 141204, 2024-03-23T13:21:46.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-03-23T13:21:46.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2024-03-23T13:21:46.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.034543582, 2024-03-23T13:21:46.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.010334688, 2024-03-23T13:21:46.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.011744377, 2024-03-23T13:21:46.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.009291407, 2024-03-23T13:21:46.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:46.562 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 454, 2024-03-23T13:21:46.562 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.010805424, 2024-03-23T13:21:46.562 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000023800 2024-03-23T13:21:46.562 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.563 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:46.563 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.563 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.563 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.563 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.563 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T13:21:46.563 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T13:21:46.563 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.563 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:21:46.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 129, 2024-03-23T13:21:46.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 185, 2024-03-23T13:21:46.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 13059109, 2024-03-23T13:21:46.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 212690, 2024-03-23T13:21:46.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 5, 2024-03-23T13:21:46.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 4, 2024-03-23T13:21:46.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.017687798, 2024-03-23T13:21:46.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.003387681, 2024-03-23T13:21:46.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.010865144, 2024-03-23T13:21:46.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.002467226, 2024-03-23T13:21:46.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:46.565 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 183, 2024-03-23T13:21:46.565 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.007231081, 2024-03-23T13:21:46.565 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000039514 2024-03-23T13:21:46.565 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:46.565 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.566 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.566 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.566 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.566 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T13:21:46.566 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 172816 2024-03-23T13:21:46.566 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.566 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:21:46.566 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 310, 2024-03-23T13:21:46.566 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 301, 2024-03-23T13:21:46.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 8924398, 2024-03-23T13:21:46.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 114290, 2024-03-23T13:21:46.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-03-23T13:21:46.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-03-23T13:21:46.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.031357181, 2024-03-23T13:21:46.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.007077786, 2024-03-23T13:21:46.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.019685814, 2024-03-23T13:21:46.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.006479357, 2024-03-23T13:21:46.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:46.568 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 301, 2024-03-23T13:21:46.568 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.009063226, 2024-03-23T13:21:46.568 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000030110 2024-03-23T13:21:46.568 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.568 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:46.568 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.568 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.568 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.568 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.569 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 481360, 2024-03-23T13:21:46.569 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 47424 2024-03-23T13:21:46.569 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.569 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-03-23T13:21:46.569 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-03-23T13:21:46.569 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-03-23T13:21:46.569 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.569 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-03-23T13:21:46.569 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-03-23T13:21:46.569 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-03-23T13:21:46.570 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1322, 2024-03-23T13:21:46.570 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.054356529, 2024-03-23T13:21:46.570 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000041116 2024-03-23T13:21:46.570 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:46.570 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.570 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-03-23T13:21:46.570 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-03-23T13:21:46.570 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-03-23T13:21:46.571 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.571 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.571 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.571 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:46.571 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.571 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-03-23T13:21:46.571 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 0, 2024-03-23T13:21:46.571 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 0, 2024-03-23T13:21:46.571 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-03-23T13:21:46.571 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.572 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.572 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.572 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.572 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-03-23T13:21:46.572 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-03-23T13:21:46.572 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-03-23T13:21:46.572 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-03-23T13:21:46.572 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 0, 2024-03-23T13:21:46.572 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-03-23T13:21:46.573 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-03-23T13:21:46.573 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 1647, 2024-03-23T13:21:46.573 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 1067, 2024-03-23T13:21:46.573 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 580, 2024-03-23T13:21:46.573 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-03-23T13:21:46.573 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 248, 2024-03-23T13:21:46.573 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-03-23T13:21:46.573 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 0, 2024-03-23T13:21:46.573 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 0, 2024-03-23T13:21:46.573 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 25, 2024-03-23T13:21:46.574 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 0, 2024-03-23T13:21:46.574 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 0, 2024-03-23T13:21:46.574 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-03-23T13:21:46.574 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-03-23T13:21:46.574 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-03-23T13:21:46.574 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-03-23T13:21:46.574 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-03-23T13:21:46.574 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-03-23T13:21:46.574 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-03-23T13:21:46.575 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-03-23T13:21:46.575 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-03-23T13:21:46.575 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-03-23T13:21:46.575 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-03-23T13:21:46.575 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-03-23T13:21:46.575 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-03-23T13:21:46.575 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-03-23T13:21:46.575 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 3452, 2024-03-23T13:21:46.575 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 26211650, 2024-03-23T13:21:46.575 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-03-23T13:21:46.576 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T13:21:46.576 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-03-23T13:21:46.576 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T13:21:46.576 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-03-23T13:21:46.576 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-03-23T13:21:46.576 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T13:21:46.576 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T13:21:46.576 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-03-23T13:21:46.576 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-03-23T13:21:46.577 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T13:21:46.577 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 0, 2024-03-23T13:21:46.577 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T13:21:46.577 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T13:21:46.577 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T13:21:46.577 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 0, 2024-03-23T13:21:46.577 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-03-23T13:21:46.577 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-03-23T13:21:46.577 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 0 2024-03-23T13:21:46.577 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.578 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-03-23T13:21:46.578 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-03-23T13:21:46.578 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-03-23T13:21:46.578 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-03-23T13:21:46.578 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-03-23T13:21:46.578 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-03-23T13:21:46.578 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-03-23T13:21:46.578 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-03-23T13:21:46.578 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-03-23T13:21:46.579 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-03-23T13:21:46.579 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-03-23T13:21:46.579 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-03-23T13:21:46.579 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-03-23T13:21:46.579 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-03-23T13:21:46.579 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-03-23T13:21:46.579 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-03-23T13:21:46.579 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-03-23T13:21:46.579 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-03-23T13:21:46.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-03-23T13:21:46.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-03-23T13:21:46.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T13:21:46.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-03-23T13:21:46.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-03-23T13:21:46.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-03-23T13:21:46.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-03-23T13:21:46.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-03-23T13:21:46.580 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.580 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-03-23T13:21:46.581 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-03-23T13:21:46.581 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 0, 2024-03-23T13:21:46.581 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-03-23T13:21:46.581 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-03-23T13:21:46.581 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 13700, 2024-03-23T13:21:46.581 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-03-23T13:21:46.581 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-03-23T13:21:46.581 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 0, 2024-03-23T13:21:46.581 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-03-23T13:21:46.582 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-03-23T13:21:46.582 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 2, 2024-03-23T13:21:46.582 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 9, 2024-03-23T13:21:46.582 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-03-23T13:21:46.582 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-03-23T13:21:46.582 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 32712541, 2024-03-23T13:21:46.582 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 33997205, 2024-03-23T13:21:46.582 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 33995745, 2024-03-23T13:21:46.582 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-03-23T13:21:46.582 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.583 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.583 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.583 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.583 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 13700 2024-03-23T13:21:46.583 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.583 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-03-23T13:21:46.583 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 69, 2024-03-23T13:21:46.583 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 5520, 2024-03-23T13:21:46.583 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 5451, 2024-03-23T13:21:46.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 44, 2024-03-23T13:21:46.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 269, 2024-03-23T13:21:46.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 225, 2024-03-23T13:21:46.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 1647, 2024-03-23T13:21:46.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 6168, 2024-03-23T13:21:46.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 4521, 2024-03-23T13:21:46.584 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 0, 2024-03-23T13:21:46.584 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 0, 2024-03-23T13:21:46.584 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 0, 2024-03-23T13:21:46.584 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 60360, 2024-03-23T13:21:46.585 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 215420 2024-03-23T13:21:46.585 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.585 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-03-23T13:21:46.585 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 0, 2024-03-23T13:21:46.585 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-03-23T13:21:46.585 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 0, 2024-03-23T13:21:46.585 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 0, 2024-03-23T13:21:46.585 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 0, 2024-03-23T13:21:46.585 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-03-23T13:21:46.585 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.586 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.586 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.586 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.586 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-03-23T13:21:46.586 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.586 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.586 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.586 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.586 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-03-23T13:21:46.587 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.587 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.587 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.587 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.587 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-03-23T13:21:46.587 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.587 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.587 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.587 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.587 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-03-23T13:21:46.588 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.588 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.588 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.588 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.588 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-03-23T13:21:46.588 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.588 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.588 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.588 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-03-23T13:21:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-03-23T13:21:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.590 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-03-23T13:21:46.590 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.590 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.590 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.590 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.590 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-03-23T13:21:46.590 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.590 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.590 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.591 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.591 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-03-23T13:21:46.591 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.591 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.591 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.591 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.592 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-03-23T13:21:46.592 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.592 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.592 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.592 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.592 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-03-23T13:21:46.592 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.592 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.593 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.593 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.593 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-03-23T13:21:46.593 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.593 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.593 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.593 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.593 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-03-23T13:21:46.593 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.594 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.594 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.594 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.594 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-03-23T13:21:46.594 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.594 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.594 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.594 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.595 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-03-23T13:21:46.595 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.595 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.595 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.595 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.595 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-03-23T13:21:46.595 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.595 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.595 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.595 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.596 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-03-23T13:21:46.596 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.596 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.596 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.596 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.596 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-03-23T13:21:46.596 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.596 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.596 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.597 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.597 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-03-23T13:21:46.597 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.597 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.597 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.597 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.597 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-03-23T13:21:46.597 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.597 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.597 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.598 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.598 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-03-23T13:21:46.598 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.598 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.598 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.598 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.598 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-03-23T13:21:46.598 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.599 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.599 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.599 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.599 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-03-23T13:21:46.599 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.599 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.599 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.599 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.599 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-03-23T13:21:46.600 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.600 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.600 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.600 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.600 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-03-23T13:21:46.600 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.600 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.600 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.600 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.600 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-03-23T13:21:46.601 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.601 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.601 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.601 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.601 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-03-23T13:21:46.601 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.601 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.601 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.601 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.602 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-03-23T13:21:46.602 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-03-23T13:21:46.602 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-03-23T13:21:46.602 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.602 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.602 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.602 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:46.602 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.602 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-03-23T13:21:46.603 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 1, 2024-03-23T13:21:46.603 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 2, 2024-03-23T13:21:46.603 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 1, 2024-03-23T13:21:46.603 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 1, 2024-03-23T13:21:46.603 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-03-23T13:21:46.603 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 0, 2024-03-23T13:21:46.603 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 0, 2024-03-23T13:21:46.603 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 1495, 2024-03-23T13:21:46.603 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-03-23T13:21:46.603 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.604 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-03-23T13:21:46.604 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:21:46.604 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-03-23T13:21:46.604 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:21:46.604 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:21:46.604 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:21:46.604 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:21:46.604 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:21:46.604 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:21:46.605 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:21:46.605 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:21:46.605 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:21:46.605 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:21:46.605 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:21:46.605 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:21:46.605 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:21:46.605 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-03-23T13:21:46.605 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:21:46.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-03-23T13:21:46.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:21:46.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:21:46.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:21:46.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:21:46.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:21:46.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:21:46.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:21:46.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-03-23T13:21:46.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:21:46.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:21:46.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:21:46.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-03-23T13:21:46.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-03-23T13:21:46.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-03-23T13:21:46.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:21:46.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:21:46.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:21:46.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:21:46.608 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 199098, 2024-03-23T13:21:46.608 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 100, 2024-03-23T13:21:46.608 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-03-23T13:21:46.608 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-03-23T13:21:46.608 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-03-23T13:21:46.608 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-03-23T13:21:46.608 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:21:46.608 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-03-23T13:21:46.609 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:21:46.609 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-03-23T13:21:46.609 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 7528, 2024-03-23T13:21:46.609 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 158, 2024-03-23T13:21:46.609 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:21:46.609 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:21:46.609 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-03-23T13:21:46.609 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-03-23T13:21:46.609 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 1375448, 2024-03-23T13:21:46.610 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 8327, 2024-03-23T13:21:46.610 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-03-23T13:21:46.610 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-03-23T13:21:46.610 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-03-23T13:21:46.610 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-03-23T13:21:46.610 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.610 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-03-23T13:21:46.610 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-03-23T13:21:46.610 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-03-23T13:21:46.611 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 797, 2024-03-23T13:21:46.611 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 26, 2024-03-23T13:21:46.611 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 4, 2024-03-23T13:21:46.611 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 797, 2024-03-23T13:21:46.611 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-03-23T13:21:46.611 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 797, 2024-03-23T13:21:46.611 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.359993226, 2024-03-23T13:21:46.611 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000451685 2024-03-23T13:21:46.612 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.612 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-03-23T13:21:46.612 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-03-23T13:21:46.612 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 797, 2024-03-23T13:21:46.612 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 798 2024-03-23T13:21:46.612 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.612 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 797, 2024-03-23T13:21:46.612 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 797, 2024-03-23T13:21:46.612 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 0, 2024-03-23T13:21:46.612 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-03-23T13:21:46.613 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-03-23T13:21:46.613 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 482, 2024-03-23T13:21:46.613 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 0, 2024-03-23T13:21:46.613 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 312, 2024-03-23T13:21:46.613 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 0, 2024-03-23T13:21:46.613 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 0, 2024-03-23T13:21:46.613 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-03-23T13:21:46.613 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-03-23T13:21:46.614 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-03-23T13:21:46.614 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-03-23T13:21:46.614 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 0, 2024-03-23T13:21:46.614 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-03-23T13:21:46.614 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-03-23T13:21:46.614 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-03-23T13:21:46.614 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 2, 2024-03-23T13:21:46.614 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 0, 2024-03-23T13:21:46.614 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-03-23T13:21:46.615 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-03-23T13:21:46.615 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-03-23T13:21:46.615 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-03-23T13:21:46.615 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-03-23T13:21:46.615 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-03-23T13:21:46.615 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T13:21:46.615 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-03-23T13:21:46.615 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-03-23T13:21:46.615 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-03-23T13:21:46.616 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-03-23T13:21:46.616 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-03-23T13:21:46.616 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-03-23T13:21:46.616 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-03-23T13:21:46.616 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-03-23T13:21:46.616 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-03-23T13:21:46.616 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-03-23T13:21:46.616 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-03-23T13:21:46.616 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-03-23T13:21:46.617 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-03-23T13:21:46.617 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-03-23T13:21:46.617 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-03-23T13:21:46.617 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-03-23T13:21:46.617 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-03-23T13:21:46.617 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-03-23T13:21:46.617 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-03-23T13:21:46.617 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 87, 2024-03-23T13:21:46.618 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-03-23T13:21:46.618 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 2, 2024-03-23T13:21:46.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 6, 2024-03-23T13:21:46.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 6, 2024-03-23T13:21:46.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-03-23T13:21:46.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-03-23T13:21:46.618 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 0, 2024-03-23T13:21:46.618 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 2, 2024-03-23T13:21:46.618 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 0 2024-03-23T13:21:46.619 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.619 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-03-23T13:21:46.619 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 0, 2024-03-23T13:21:46.619 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-03-23T13:21:46.619 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-03-23T13:21:46.619 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-03-23T13:21:46.619 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.619 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-03-23T13:21:46.619 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-03-23T13:21:46.620 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.620 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-03-23T13:21:46.620 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-03-23T13:21:46.620 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T13:21:46.620 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-03-23T13:21:46.620 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-03-23T13:21:46.620 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-03-23T13:21:46.620 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-03-23T13:21:46.620 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-03-23T13:21:46.621 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.621 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:21:46.621 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:46.621 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-03-23T13:21:46.621 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:46.621 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 889, 2024-03-23T13:21:46.621 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 30423627, 2024-03-23T13:21:46.621 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:46.622 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 889, 2024-03-23T13:21:46.622 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:46.622 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:46.622 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 889, 2024-03-23T13:21:46.622 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 30423627, 2024-03-23T13:21:46.622 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:46.622 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.622 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.622 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.623 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:46.623 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.623 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-03-23T13:21:46.623 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:46.623 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-03-23T13:21:46.623 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:46.623 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:46.623 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:46.623 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:46.624 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:46.624 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 797, 2024-03-23T13:21:46.624 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 29806371, 2024-03-23T13:21:46.624 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 797, 2024-03-23T13:21:46.624 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 29806371, 2024-03-23T13:21:46.624 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:46.624 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.624 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.625 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.625 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:46.625 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.625 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-03-23T13:21:46.625 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:46.625 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-03-23T13:21:46.625 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:46.625 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:46.625 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:46.626 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:46.626 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:46.626 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 797, 2024-03-23T13:21:46.626 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 797, 2024-03-23T13:21:46.626 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 797, 2024-03-23T13:21:46.626 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 797, 2024-03-23T13:21:46.626 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:46.626 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.627 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.627 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.627 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:46.627 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.627 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-03-23T13:21:46.627 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:46.627 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-03-23T13:21:46.627 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:46.628 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:46.628 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:46.628 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:46.628 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:46.628 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:46.628 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:46.628 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-03-23T13:21:46.628 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-03-23T13:21:46.628 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:46.629 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.629 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.629 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.629 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:46.629 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:46.629 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x55d39fd8e6e0": { 2024-03-23T13:21:46.629 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:46.629 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-03-23T13:21:46.630 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:46.630 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:46.630 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:46.630 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:46.630 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:46.630 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:46.630 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:46.631 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-03-23T13:21:46.631 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-03-23T13:21:46.631 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:46.631 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:46.631 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:46.631 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:46.631 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:46.631 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:46.631 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-03-23T13:21:47.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:47 smithi007 ceph-mon[32184]: pgmap v409: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 169 KiB/s wr, 40 op/s 2024-03-23T13:21:47.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:47 smithi130 ceph-mon[36225]: pgmap v409: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 169 KiB/s wr, 40 op/s 2024-03-23T13:21:47.748 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-23T13:21:47.749 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-23T13:21:47.749 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-03-23T13:21:47.749 DEBUG:tasks.check_counter:Getting stats from h 2024-03-23T13:21:47.749 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-23T13:21:47.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:47 smithi155 ceph-mon[36157]: pgmap v409: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 169 KiB/s wr, 40 op/s 2024-03-23T13:21:48.033 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:21:49.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:49 smithi130 ceph-mon[36225]: pgmap v410: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 36 KiB/s wr, 24 op/s 2024-03-23T13:21:49.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:49 smithi007 ceph-mon[32184]: pgmap v410: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 36 KiB/s wr, 24 op/s 2024-03-23T13:21:49.591 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:49 smithi155 ceph-mon[36157]: pgmap v410: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 36 KiB/s wr, 24 op/s 2024-03-23T13:21:49.793 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-03-23T13:21:49.793 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:21:49.794 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1978, 2024-03-23T13:21:49.794 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 3447, 2024-03-23T13:21:49.794 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1821332, 2024-03-23T13:21:49.794 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 5040181, 2024-03-23T13:21:49.794 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 11, 2024-03-23T13:21:49.794 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 4, 2024-03-23T13:21:49.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.173868218, 2024-03-23T13:21:49.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.083189600, 2024-03-23T13:21:49.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.086828704, 2024-03-23T13:21:49.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.009202104, 2024-03-23T13:21:49.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:49.795 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3450, 2024-03-23T13:21:49.795 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.335496120, 2024-03-23T13:21:49.795 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000097245 2024-03-23T13:21:49.795 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.796 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:49.796 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 4638, 2024-03-23T13:21:49.796 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000893988, 2024-03-23T13:21:49.796 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000192 2024-03-23T13:21:49.796 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.796 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-23T13:21:49.796 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2024-03-23T13:21:49.796 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.797 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:21:49.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 254, 2024-03-23T13:21:49.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 309, 2024-03-23T13:21:49.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 76882, 2024-03-23T13:21:49.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 282942, 2024-03-23T13:21:49.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 10, 2024-03-23T13:21:49.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 7, 2024-03-23T13:21:49.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.020886823, 2024-03-23T13:21:49.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.005353302, 2024-03-23T13:21:49.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.009261347, 2024-03-23T13:21:49.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.004493240, 2024-03-23T13:21:49.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:49.798 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 307, 2024-03-23T13:21:49.798 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.029928057, 2024-03-23T13:21:49.798 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000097485 2024-03-23T13:21:49.798 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.799 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:49.799 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 121, 2024-03-23T13:21:49.799 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000009560, 2024-03-23T13:21:49.799 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000079 2024-03-23T13:21:49.799 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.799 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T13:21:49.799 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 174256 2024-03-23T13:21:49.799 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.799 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:21:49.800 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 2360, 2024-03-23T13:21:49.800 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 4847, 2024-03-23T13:21:49.800 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1229352, 2024-03-23T13:21:49.800 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 2453740, 2024-03-23T13:21:49.800 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 13, 2024-03-23T13:21:49.800 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 7, 2024-03-23T13:21:49.800 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.166952845, 2024-03-23T13:21:49.800 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.070780798, 2024-03-23T13:21:49.801 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.162865660, 2024-03-23T13:21:49.801 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.005421377, 2024-03-23T13:21:49.801 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:49.801 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 4842, 2024-03-23T13:21:49.801 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.348069553, 2024-03-23T13:21:49.801 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000071885 2024-03-23T13:21:49.801 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.801 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:49.801 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 6063, 2024-03-23T13:21:49.802 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.001007725, 2024-03-23T13:21:49.802 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000166 2024-03-23T13:21:49.802 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.802 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 492176, 2024-03-23T13:21:49.802 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 50176 2024-03-23T13:21:49.802 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.802 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-03-23T13:21:49.802 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-03-23T13:21:49.802 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-03-23T13:21:49.803 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.803 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-03-23T13:21:49.803 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-03-23T13:21:49.803 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-03-23T13:21:49.803 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 209, 2024-03-23T13:21:49.803 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.759495028, 2024-03-23T13:21:49.803 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.003633947 2024-03-23T13:21:49.803 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:49.803 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.804 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-03-23T13:21:49.804 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-03-23T13:21:49.804 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-03-23T13:21:49.804 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 769, 2024-03-23T13:21:49.804 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.053791626, 2024-03-23T13:21:49.804 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000069950 2024-03-23T13:21:49.804 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:49.804 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.805 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-03-23T13:21:49.805 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 1271, 2024-03-23T13:21:49.805 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 1261, 2024-03-23T13:21:49.805 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-03-23T13:21:49.805 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1261, 2024-03-23T13:21:49.805 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 1.683701464, 2024-03-23T13:21:49.805 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001335211 2024-03-23T13:21:49.805 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.805 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-03-23T13:21:49.806 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 10, 2024-03-23T13:21:49.806 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-03-23T13:21:49.806 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-03-23T13:21:49.806 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 11, 2024-03-23T13:21:49.806 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-03-23T13:21:49.806 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-03-23T13:21:49.806 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 1178, 2024-03-23T13:21:49.806 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 0, 2024-03-23T13:21:49.806 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 1178, 2024-03-23T13:21:49.807 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-03-23T13:21:49.807 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 1096, 2024-03-23T13:21:49.807 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-03-23T13:21:49.807 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 187, 2024-03-23T13:21:49.807 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 187, 2024-03-23T13:21:49.807 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 64, 2024-03-23T13:21:49.807 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 2631, 2024-03-23T13:21:49.807 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 1787, 2024-03-23T13:21:49.807 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 9, 2024-03-23T13:21:49.808 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 39, 2024-03-23T13:21:49.808 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-03-23T13:21:49.808 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-03-23T13:21:49.808 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-03-23T13:21:49.808 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 2845, 2024-03-23T13:21:49.808 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-03-23T13:21:49.808 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-03-23T13:21:49.808 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-03-23T13:21:49.808 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 60, 2024-03-23T13:21:49.808 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 1146, 2024-03-23T13:21:49.809 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-03-23T13:21:49.809 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-03-23T13:21:49.809 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 1, 2024-03-23T13:21:49.809 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 3452, 2024-03-23T13:21:49.809 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 26211650, 2024-03-23T13:21:49.809 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-03-23T13:21:49.809 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T13:21:49.809 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-03-23T13:21:49.809 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T13:21:49.810 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-03-23T13:21:49.810 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-03-23T13:21:49.810 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T13:21:49.810 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T13:21:49.810 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-03-23T13:21:49.810 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-03-23T13:21:49.810 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 6, 2024-03-23T13:21:49.810 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 730, 2024-03-23T13:21:49.811 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T13:21:49.811 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T13:21:49.811 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T13:21:49.811 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 784, 2024-03-23T13:21:49.811 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 775, 2024-03-23T13:21:49.811 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 11, 2024-03-23T13:21:49.811 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 773 2024-03-23T13:21:49.811 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.811 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-03-23T13:21:49.812 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-03-23T13:21:49.812 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-03-23T13:21:49.812 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-03-23T13:21:49.812 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 23, 2024-03-23T13:21:49.812 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 60, 2024-03-23T13:21:49.812 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-03-23T13:21:49.812 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-03-23T13:21:49.812 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-03-23T13:21:49.813 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 415, 2024-03-23T13:21:49.813 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 415, 2024-03-23T13:21:49.813 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-03-23T13:21:49.813 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-03-23T13:21:49.813 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-03-23T13:21:49.813 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-03-23T13:21:49.813 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-03-23T13:21:49.813 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-03-23T13:21:49.813 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-03-23T13:21:49.814 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-03-23T13:21:49.814 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-03-23T13:21:49.814 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T13:21:49.814 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-03-23T13:21:49.814 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-03-23T13:21:49.814 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-03-23T13:21:49.814 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-03-23T13:21:49.815 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-03-23T13:21:49.815 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.815 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-03-23T13:21:49.815 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-03-23T13:21:49.815 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 2552, 2024-03-23T13:21:49.815 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-03-23T13:21:49.815 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-03-23T13:21:49.815 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 2552, 2024-03-23T13:21:49.816 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-03-23T13:21:49.816 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-03-23T13:21:49.816 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 4, 2024-03-23T13:21:49.816 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-03-23T13:21:49.816 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-03-23T13:21:49.816 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 4, 2024-03-23T13:21:49.816 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 2, 2024-03-23T13:21:49.816 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-03-23T13:21:49.817 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-03-23T13:21:49.817 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-03-23T13:21:49.817 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 8366057, 2024-03-23T13:21:49.817 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 0, 2024-03-23T13:21:49.817 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-03-23T13:21:49.817 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 81, 2024-03-23T13:21:49.817 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.194174749, 2024-03-23T13:21:49.817 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.002397219 2024-03-23T13:21:49.817 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.818 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 0 2024-03-23T13:21:49.818 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.818 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-03-23T13:21:49.818 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 981, 2024-03-23T13:21:49.818 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 1396, 2024-03-23T13:21:49.818 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 415, 2024-03-23T13:21:49.818 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 85, 2024-03-23T13:21:49.818 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 86, 2024-03-23T13:21:49.819 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 1, 2024-03-23T13:21:49.819 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 1178, 2024-03-23T13:21:49.819 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 1593, 2024-03-23T13:21:49.819 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 415, 2024-03-23T13:21:49.819 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 187, 2024-03-23T13:21:49.819 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 1369, 2024-03-23T13:21:49.819 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 1182, 2024-03-23T13:21:49.819 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 66184, 2024-03-23T13:21:49.819 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 223612 2024-03-23T13:21:49.820 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.820 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-03-23T13:21:49.820 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 2175, 2024-03-23T13:21:49.820 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-03-23T13:21:49.820 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 1271, 2024-03-23T13:21:49.820 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 2, 2024-03-23T13:21:49.820 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 725, 2024-03-23T13:21:49.820 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-03-23T13:21:49.821 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 310, 2024-03-23T13:21:49.821 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.578126641, 2024-03-23T13:21:49.821 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001864924 2024-03-23T13:21:49.821 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.821 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-03-23T13:21:49.821 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.821 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.821 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.822 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.822 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-03-23T13:21:49.822 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.822 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.822 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.822 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.822 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-03-23T13:21:49.822 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.823 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.823 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.823 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.823 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-03-23T13:21:49.823 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 536, 2024-03-23T13:21:49.823 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.196603125, 2024-03-23T13:21:49.823 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000366796 2024-03-23T13:21:49.823 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.823 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-03-23T13:21:49.824 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.824 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.824 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.824 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.824 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-03-23T13:21:49.824 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.824 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.824 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.824 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.825 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-03-23T13:21:49.825 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.825 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.825 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.825 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.825 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-03-23T13:21:49.825 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.825 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.826 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.826 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.826 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-03-23T13:21:49.826 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.826 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.826 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.826 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.826 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-03-23T13:21:49.826 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.827 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.827 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.827 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.827 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-03-23T13:21:49.827 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.828 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.828 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.828 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.828 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-03-23T13:21:49.828 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.828 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.828 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.828 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.828 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-03-23T13:21:49.829 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.829 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.829 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.829 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.829 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-03-23T13:21:49.829 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.829 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.829 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.830 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.830 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-03-23T13:21:49.830 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.830 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.830 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.830 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.830 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-03-23T13:21:49.830 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.830 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.831 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.831 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.831 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-03-23T13:21:49.831 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.831 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.831 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.831 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.831 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-03-23T13:21:49.831 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.832 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.832 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.832 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.832 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-03-23T13:21:49.832 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.832 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.832 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.832 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.832 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-03-23T13:21:49.832 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.833 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.833 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.833 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.833 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-03-23T13:21:49.833 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.833 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.833 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.833 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.833 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-03-23T13:21:49.834 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.834 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.834 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.834 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.834 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-03-23T13:21:49.834 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.834 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.834 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.834 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.835 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-03-23T13:21:49.835 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.835 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.835 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.835 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.835 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-03-23T13:21:49.835 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.835 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.836 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.836 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.836 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-03-23T13:21:49.836 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.836 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.836 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.836 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.836 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-03-23T13:21:49.836 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.837 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.837 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.837 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.837 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-03-23T13:21:49.837 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 415, 2024-03-23T13:21:49.837 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.908971698, 2024-03-23T13:21:49.837 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.002190293 2024-03-23T13:21:49.837 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.837 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-03-23T13:21:49.837 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-03-23T13:21:49.838 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-03-23T13:21:49.838 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.838 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.838 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.838 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:49.838 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.838 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-03-23T13:21:49.838 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 1, 2024-03-23T13:21:49.838 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 1, 2024-03-23T13:21:49.839 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 0, 2024-03-23T13:21:49.839 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 1, 2024-03-23T13:21:49.839 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-03-23T13:21:49.839 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 1153, 2024-03-23T13:21:49.839 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 1153, 2024-03-23T13:21:49.839 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 124, 2024-03-23T13:21:49.839 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-03-23T13:21:49.839 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.839 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-03-23T13:21:49.840 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:21:49.840 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-03-23T13:21:49.840 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:21:49.840 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:21:49.840 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:21:49.840 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:21:49.840 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:21:49.840 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:21:49.840 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:21:49.841 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:21:49.841 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:21:49.841 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:21:49.841 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:21:49.841 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:21:49.841 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:21:49.841 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-03-23T13:21:49.841 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:21:49.841 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-03-23T13:21:49.841 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:21:49.842 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:21:49.842 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:21:49.842 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:21:49.842 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:21:49.842 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:21:49.842 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:21:49.842 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-03-23T13:21:49.842 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:21:49.842 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:21:49.843 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:21:49.843 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-03-23T13:21:49.843 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-03-23T13:21:49.843 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-03-23T13:21:49.843 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:21:49.843 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:21:49.843 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:21:49.843 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:21:49.843 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 318594, 2024-03-23T13:21:49.844 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 1083, 2024-03-23T13:21:49.844 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 880, 2024-03-23T13:21:49.844 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 10, 2024-03-23T13:21:49.844 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-03-23T13:21:49.844 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-03-23T13:21:49.844 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:21:49.844 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-03-23T13:21:49.844 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:21:49.844 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-03-23T13:21:49.844 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 7528, 2024-03-23T13:21:49.845 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 158, 2024-03-23T13:21:49.845 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:21:49.845 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:21:49.845 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-03-23T13:21:49.845 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-03-23T13:21:49.845 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 3340510, 2024-03-23T13:21:49.845 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 37666, 2024-03-23T13:21:49.845 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-03-23T13:21:49.845 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-03-23T13:21:49.846 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-03-23T13:21:49.846 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-03-23T13:21:49.846 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.846 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-03-23T13:21:49.846 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-03-23T13:21:49.846 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-03-23T13:21:49.846 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 593, 2024-03-23T13:21:49.846 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 4315249, 2024-03-23T13:21:49.846 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 28, 2024-03-23T13:21:49.847 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 593, 2024-03-23T13:21:49.847 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-03-23T13:21:49.847 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 593, 2024-03-23T13:21:49.847 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2.020962031, 2024-03-23T13:21:49.847 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.003408030 2024-03-23T13:21:49.847 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.847 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-03-23T13:21:49.847 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-03-23T13:21:49.848 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 593, 2024-03-23T13:21:49.848 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 636 2024-03-23T13:21:49.848 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.848 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 593, 2024-03-23T13:21:49.848 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 14, 2024-03-23T13:21:49.848 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 579, 2024-03-23T13:21:49.848 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-03-23T13:21:49.848 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-03-23T13:21:49.849 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 0, 2024-03-23T13:21:49.849 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 10, 2024-03-23T13:21:49.849 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 14, 2024-03-23T13:21:49.849 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 95, 2024-03-23T13:21:49.849 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 15, 2024-03-23T13:21:49.849 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-03-23T13:21:49.849 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-03-23T13:21:49.849 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-03-23T13:21:49.850 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-03-23T13:21:49.850 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 427, 2024-03-23T13:21:49.850 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-03-23T13:21:49.850 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-03-23T13:21:49.850 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-03-23T13:21:49.850 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 0, 2024-03-23T13:21:49.850 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 20, 2024-03-23T13:21:49.851 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-03-23T13:21:49.851 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-03-23T13:21:49.851 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-03-23T13:21:49.851 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-03-23T13:21:49.851 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-03-23T13:21:49.851 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-03-23T13:21:49.851 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T13:21:49.851 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-03-23T13:21:49.852 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-03-23T13:21:49.852 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-03-23T13:21:49.852 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-03-23T13:21:49.852 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-03-23T13:21:49.852 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-03-23T13:21:49.852 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-03-23T13:21:49.852 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-03-23T13:21:49.852 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-03-23T13:21:49.853 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-03-23T13:21:49.853 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-03-23T13:21:49.853 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-03-23T13:21:49.853 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-03-23T13:21:49.853 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-03-23T13:21:49.853 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-03-23T13:21:49.853 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-03-23T13:21:49.853 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-03-23T13:21:49.853 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-03-23T13:21:49.854 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-03-23T13:21:49.854 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 87, 2024-03-23T13:21:49.854 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-03-23T13:21:49.854 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 2, 2024-03-23T13:21:49.854 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 12, 2024-03-23T13:21:49.854 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 12, 2024-03-23T13:21:49.854 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-03-23T13:21:49.854 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-03-23T13:21:49.855 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 47, 2024-03-23T13:21:49.855 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 0, 2024-03-23T13:21:49.855 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 8 2024-03-23T13:21:49.855 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.855 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-03-23T13:21:49.855 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 1, 2024-03-23T13:21:49.855 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-03-23T13:21:49.855 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-03-23T13:21:49.856 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-03-23T13:21:49.856 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.856 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-03-23T13:21:49.856 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-03-23T13:21:49.856 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.856 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-03-23T13:21:49.856 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-03-23T13:21:49.856 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 6, 2024-03-23T13:21:49.856 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-03-23T13:21:49.857 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 6, 2024-03-23T13:21:49.857 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 415, 2024-03-23T13:21:49.857 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 415, 2024-03-23T13:21:49.857 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-03-23T13:21:49.857 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.857 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:21:49.857 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:49.857 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-03-23T13:21:49.857 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:49.857 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 4588, 2024-03-23T13:21:49.858 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 2705818, 2024-03-23T13:21:49.858 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:49.858 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 4588, 2024-03-23T13:21:49.858 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:49.858 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:49.858 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 4588, 2024-03-23T13:21:49.858 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2705818, 2024-03-23T13:21:49.858 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:49.859 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.859 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.859 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.859 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:49.859 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.859 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-03-23T13:21:49.859 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:49.859 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-03-23T13:21:49.859 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:49.859 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:49.860 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:49.860 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:49.860 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:49.860 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 593, 2024-03-23T13:21:49.860 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 4357164, 2024-03-23T13:21:49.860 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 593, 2024-03-23T13:21:49.860 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 4357164, 2024-03-23T13:21:49.860 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:49.860 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.861 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.861 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.861 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:49.861 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.861 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-03-23T13:21:49.861 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:49.861 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-03-23T13:21:49.861 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:49.861 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:49.861 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:49.862 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:49.862 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:49.862 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 593, 2024-03-23T13:21:49.862 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 593, 2024-03-23T13:21:49.862 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 593, 2024-03-23T13:21:49.862 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 593, 2024-03-23T13:21:49.862 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:49.862 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.863 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.863 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.863 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:49.863 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.863 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-03-23T13:21:49.863 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:49.863 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-03-23T13:21:49.863 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:49.864 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 415, 2024-03-23T13:21:49.864 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 41915, 2024-03-23T13:21:49.864 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:49.864 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 415, 2024-03-23T13:21:49.864 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:49.864 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:49.864 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 14, 2024-03-23T13:21:49.864 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 41915, 2024-03-23T13:21:49.864 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:49.865 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.865 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.865 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.865 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:49.865 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:49.865 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x558cd8538500": { 2024-03-23T13:21:49.865 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:49.865 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-03-23T13:21:49.865 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:49.866 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 2552, 2024-03-23T13:21:49.866 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 4171753, 2024-03-23T13:21:49.866 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:49.866 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 2552, 2024-03-23T13:21:49.866 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:49.866 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:49.866 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 81, 2024-03-23T13:21:49.866 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 4171753, 2024-03-23T13:21:49.867 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:49.867 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:49.867 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:49.867 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:49.867 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:49.867 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:49.867 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-03-23T13:21:50.636 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-23T13:21:50.636 INFO:tasks.check_counter:Daemon mds.h mds.imported=60 2024-03-23T13:21:50.636 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-03-23T13:21:50.636 DEBUG:tasks.check_counter:Getting stats from k 2024-03-23T13:21:50.636 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-23T13:21:50.877 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T13:21:51.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:51 smithi007 ceph-mon[32184]: pgmap v411: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-03-23T13:21:51.642 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:51 smithi130 ceph-mon[36225]: pgmap v411: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-03-23T13:21:51.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:51 smithi155 ceph-mon[36157]: pgmap v411: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-03-23T13:21:52.558 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-03-23T13:21:52.558 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:21:52.558 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 99, 2024-03-23T13:21:52.558 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 90, 2024-03-23T13:21:52.559 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 483122, 2024-03-23T13:21:52.559 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 50141, 2024-03-23T13:21:52.559 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 5, 2024-03-23T13:21:52.559 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2024-03-23T13:21:52.559 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.013263984, 2024-03-23T13:21:52.559 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.003311715, 2024-03-23T13:21:52.559 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.010502746, 2024-03-23T13:21:52.560 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.001411871, 2024-03-23T13:21:52.560 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:52.560 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 90, 2024-03-23T13:21:52.560 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.004271367, 2024-03-23T13:21:52.560 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000047459 2024-03-23T13:21:52.560 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.560 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:52.560 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.560 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.561 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.561 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 482288, 2024-03-23T13:21:52.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 49824 2024-03-23T13:21:52.561 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.561 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:21:52.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 163, 2024-03-23T13:21:52.561 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 222, 2024-03-23T13:21:52.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 40034, 2024-03-23T13:21:52.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 226963, 2024-03-23T13:21:52.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 5, 2024-03-23T13:21:52.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 4, 2024-03-23T13:21:52.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.014407095, 2024-03-23T13:21:52.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.004822899, 2024-03-23T13:21:52.562 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.003747401, 2024-03-23T13:21:52.563 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.004427051, 2024-03-23T13:21:52.563 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:52.563 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 220, 2024-03-23T13:21:52.563 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.007529490, 2024-03-23T13:21:52.563 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000034224 2024-03-23T13:21:52.563 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.563 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:52.563 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.564 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.564 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.564 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T13:21:52.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 176576 2024-03-23T13:21:52.564 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.564 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:21:52.564 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 511, 2024-03-23T13:21:52.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 513, 2024-03-23T13:21:52.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 4345973, 2024-03-23T13:21:52.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 158979, 2024-03-23T13:21:52.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-03-23T13:21:52.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-03-23T13:21:52.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.034659312, 2024-03-23T13:21:52.565 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.010663770, 2024-03-23T13:21:52.566 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.010765735, 2024-03-23T13:21:52.566 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.010538042, 2024-03-23T13:21:52.566 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:52.566 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 511, 2024-03-23T13:21:52.566 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.013303480, 2024-03-23T13:21:52.566 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000026034 2024-03-23T13:21:52.566 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.566 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:52.567 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.567 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.567 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.567 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T13:21:52.567 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T13:21:52.567 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.567 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-03-23T13:21:52.567 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-03-23T13:21:52.568 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-03-23T13:21:52.568 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.568 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-03-23T13:21:52.568 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-03-23T13:21:52.568 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-03-23T13:21:52.568 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1171, 2024-03-23T13:21:52.568 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.023344528, 2024-03-23T13:21:52.568 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000019935 2024-03-23T13:21:52.568 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:52.569 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.569 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-03-23T13:21:52.569 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-03-23T13:21:52.569 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-03-23T13:21:52.569 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.569 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.569 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.569 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:52.569 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.570 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-03-23T13:21:52.570 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 0, 2024-03-23T13:21:52.570 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 0, 2024-03-23T13:21:52.570 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-03-23T13:21:52.570 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.570 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.570 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.570 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.571 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-03-23T13:21:52.571 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-03-23T13:21:52.571 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-03-23T13:21:52.571 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-03-23T13:21:52.571 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 0, 2024-03-23T13:21:52.571 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-03-23T13:21:52.571 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-03-23T13:21:52.571 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 1573, 2024-03-23T13:21:52.571 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 118, 2024-03-23T13:21:52.572 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 1455, 2024-03-23T13:21:52.572 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-03-23T13:21:52.572 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 1410, 2024-03-23T13:21:52.572 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-03-23T13:21:52.572 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 0, 2024-03-23T13:21:52.572 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 0, 2024-03-23T13:21:52.572 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 120, 2024-03-23T13:21:52.572 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 0, 2024-03-23T13:21:52.572 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 0, 2024-03-23T13:21:52.573 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-03-23T13:21:52.573 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-03-23T13:21:52.573 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-03-23T13:21:52.573 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-03-23T13:21:52.573 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-03-23T13:21:52.573 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-03-23T13:21:52.573 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-03-23T13:21:52.573 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-03-23T13:21:52.573 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-03-23T13:21:52.574 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-03-23T13:21:52.574 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-03-23T13:21:52.574 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-03-23T13:21:52.574 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-03-23T13:21:52.574 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-03-23T13:21:52.574 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 0, 2024-03-23T13:21:52.574 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 0, 2024-03-23T13:21:52.574 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-03-23T13:21:52.575 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T13:21:52.575 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-03-23T13:21:52.575 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T13:21:52.575 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-03-23T13:21:52.575 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-03-23T13:21:52.575 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T13:21:52.575 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T13:21:52.576 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-03-23T13:21:52.576 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-03-23T13:21:52.576 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T13:21:52.576 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 0, 2024-03-23T13:21:52.576 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T13:21:52.576 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T13:21:52.576 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T13:21:52.576 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 0, 2024-03-23T13:21:52.576 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-03-23T13:21:52.577 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-03-23T13:21:52.577 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 0 2024-03-23T13:21:52.577 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.577 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-03-23T13:21:52.577 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-03-23T13:21:52.577 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-03-23T13:21:52.577 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-03-23T13:21:52.577 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-03-23T13:21:52.578 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-03-23T13:21:52.578 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-03-23T13:21:52.578 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-03-23T13:21:52.578 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-03-23T13:21:52.578 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-03-23T13:21:52.578 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-03-23T13:21:52.578 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-03-23T13:21:52.578 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-03-23T13:21:52.579 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-03-23T13:21:52.579 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-03-23T13:21:52.579 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-03-23T13:21:52.579 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-03-23T13:21:52.579 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-03-23T13:21:52.579 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-03-23T13:21:52.579 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-03-23T13:21:52.579 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T13:21:52.579 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-03-23T13:21:52.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-03-23T13:21:52.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-03-23T13:21:52.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-03-23T13:21:52.580 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-03-23T13:21:52.580 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.580 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-03-23T13:21:52.580 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-03-23T13:21:52.580 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 0, 2024-03-23T13:21:52.581 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-03-23T13:21:52.581 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-03-23T13:21:52.581 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 2458, 2024-03-23T13:21:52.581 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-03-23T13:21:52.581 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-03-23T13:21:52.581 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 0, 2024-03-23T13:21:52.581 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-03-23T13:21:52.581 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-03-23T13:21:52.581 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 4, 2024-03-23T13:21:52.582 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 2, 2024-03-23T13:21:52.582 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-03-23T13:21:52.582 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-03-23T13:21:52.582 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-03-23T13:21:52.582 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 8387159, 2024-03-23T13:21:52.582 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 8384548, 2024-03-23T13:21:52.582 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-03-23T13:21:52.582 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.583 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.583 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.583 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.583 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 2458 2024-03-23T13:21:52.583 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.583 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-03-23T13:21:52.583 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 856, 2024-03-23T13:21:52.583 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 1280, 2024-03-23T13:21:52.583 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 424, 2024-03-23T13:21:52.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 178, 2024-03-23T13:21:52.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 178, 2024-03-23T13:21:52.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 0, 2024-03-23T13:21:52.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 1573, 2024-03-23T13:21:52.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 1573, 2024-03-23T13:21:52.584 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 0, 2024-03-23T13:21:52.584 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 0, 2024-03-23T13:21:52.585 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 0, 2024-03-23T13:21:52.585 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 0, 2024-03-23T13:21:52.585 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 53608, 2024-03-23T13:21:52.585 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 215420 2024-03-23T13:21:52.585 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.585 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-03-23T13:21:52.585 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 0, 2024-03-23T13:21:52.585 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-03-23T13:21:52.586 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 0, 2024-03-23T13:21:52.586 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 0, 2024-03-23T13:21:52.586 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 0, 2024-03-23T13:21:52.586 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-03-23T13:21:52.586 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.586 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.586 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.587 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.587 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-03-23T13:21:52.587 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.587 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.587 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.587 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.587 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-03-23T13:21:52.587 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.587 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.588 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.588 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.588 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-03-23T13:21:52.588 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.588 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.588 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.588 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.588 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-03-23T13:21:52.588 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.589 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.589 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.589 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.589 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-03-23T13:21:52.589 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.589 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.589 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.589 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.589 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-03-23T13:21:52.590 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.590 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.590 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.590 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.590 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-03-23T13:21:52.590 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.590 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.590 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.591 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.591 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-03-23T13:21:52.591 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.591 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.591 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.591 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.591 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-03-23T13:21:52.591 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.592 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.592 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.592 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.592 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-03-23T13:21:52.592 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.592 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.592 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.592 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.592 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-03-23T13:21:52.593 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.593 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.593 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.593 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.594 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-03-23T13:21:52.594 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.594 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.594 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.594 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.594 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-03-23T13:21:52.594 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.594 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.595 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.595 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.595 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-03-23T13:21:52.595 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.595 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.595 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.595 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.595 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-03-23T13:21:52.596 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.596 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.596 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.596 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.596 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-03-23T13:21:52.596 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.596 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.597 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.597 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.597 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-03-23T13:21:52.597 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.597 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.597 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.597 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.597 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-03-23T13:21:52.597 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.598 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.598 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.598 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.598 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-03-23T13:21:52.598 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.598 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.598 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.598 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.599 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-03-23T13:21:52.599 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.599 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.599 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.599 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.599 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-03-23T13:21:52.599 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.599 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.600 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.600 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.600 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-03-23T13:21:52.600 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.600 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.600 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.600 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.600 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-03-23T13:21:52.600 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.601 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.601 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.601 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.601 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-03-23T13:21:52.601 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.601 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.601 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.601 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.601 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-03-23T13:21:52.602 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.602 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.602 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.602 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.602 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-03-23T13:21:52.602 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.602 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.602 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.603 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.603 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-03-23T13:21:52.603 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.603 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.603 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.603 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.603 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-03-23T13:21:52.603 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.603 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.604 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.604 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.604 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-03-23T13:21:52.604 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-03-23T13:21:52.604 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-03-23T13:21:52.604 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.604 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.604 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.604 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:52.605 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.605 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-03-23T13:21:52.605 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 1, 2024-03-23T13:21:52.605 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 2, 2024-03-23T13:21:52.605 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 1, 2024-03-23T13:21:52.605 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 1, 2024-03-23T13:21:52.605 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-03-23T13:21:52.605 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 0, 2024-03-23T13:21:52.606 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 0, 2024-03-23T13:21:52.606 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 1501, 2024-03-23T13:21:52.606 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-03-23T13:21:52.606 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.606 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-03-23T13:21:52.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:21:52.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-03-23T13:21:52.606 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:21:52.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:21:52.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:21:52.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:21:52.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:21:52.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:21:52.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:21:52.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:21:52.607 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:21:52.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:21:52.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:21:52.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:21:52.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:21:52.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-03-23T13:21:52.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:21:52.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-03-23T13:21:52.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:21:52.608 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:21:52.609 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:21:52.609 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:21:52.609 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:21:52.609 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:21:52.609 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:21:52.609 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-03-23T13:21:52.609 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:21:52.609 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:21:52.609 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:21:52.610 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-03-23T13:21:52.610 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-03-23T13:21:52.610 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-03-23T13:21:52.610 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:21:52.610 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:21:52.610 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:21:52.610 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:21:52.610 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 220578, 2024-03-23T13:21:52.610 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 895, 2024-03-23T13:21:52.611 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-03-23T13:21:52.611 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-03-23T13:21:52.611 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-03-23T13:21:52.611 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-03-23T13:21:52.611 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:21:52.611 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-03-23T13:21:52.611 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:21:52.611 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-03-23T13:21:52.611 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 7528, 2024-03-23T13:21:52.612 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 158, 2024-03-23T13:21:52.612 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:21:52.612 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:21:52.612 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-03-23T13:21:52.612 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-03-23T13:21:52.612 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 3297217, 2024-03-23T13:21:52.612 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 26129, 2024-03-23T13:21:52.612 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-03-23T13:21:52.613 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-03-23T13:21:52.613 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-03-23T13:21:52.613 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-03-23T13:21:52.613 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.613 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-03-23T13:21:52.613 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-03-23T13:21:52.613 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-03-23T13:21:52.613 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 674, 2024-03-23T13:21:52.613 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 21, 2024-03-23T13:21:52.614 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 3, 2024-03-23T13:21:52.614 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 674, 2024-03-23T13:21:52.614 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-03-23T13:21:52.614 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 674, 2024-03-23T13:21:52.614 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.300994330, 2024-03-23T13:21:52.614 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000446579 2024-03-23T13:21:52.614 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.614 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-03-23T13:21:52.615 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-03-23T13:21:52.615 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 674, 2024-03-23T13:21:52.615 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 675 2024-03-23T13:21:52.615 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.615 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 674, 2024-03-23T13:21:52.615 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 674, 2024-03-23T13:21:52.615 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 0, 2024-03-23T13:21:52.615 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-03-23T13:21:52.615 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-03-23T13:21:52.616 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 404, 2024-03-23T13:21:52.616 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 0, 2024-03-23T13:21:52.616 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 268, 2024-03-23T13:21:52.616 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 0, 2024-03-23T13:21:52.616 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 0, 2024-03-23T13:21:52.616 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-03-23T13:21:52.616 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-03-23T13:21:52.616 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-03-23T13:21:52.616 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-03-23T13:21:52.617 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 0, 2024-03-23T13:21:52.617 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-03-23T13:21:52.617 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-03-23T13:21:52.617 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-03-23T13:21:52.617 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 1, 2024-03-23T13:21:52.617 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 0, 2024-03-23T13:21:52.617 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-03-23T13:21:52.617 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-03-23T13:21:52.617 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-03-23T13:21:52.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-03-23T13:21:52.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-03-23T13:21:52.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-03-23T13:21:52.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T13:21:52.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-03-23T13:21:52.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-03-23T13:21:52.618 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-03-23T13:21:52.618 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-03-23T13:21:52.619 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-03-23T13:21:52.619 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-03-23T13:21:52.619 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-03-23T13:21:52.619 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-03-23T13:21:52.619 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-03-23T13:21:52.619 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-03-23T13:21:52.619 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-03-23T13:21:52.619 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-03-23T13:21:52.620 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-03-23T13:21:52.620 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-03-23T13:21:52.620 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-03-23T13:21:52.620 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-03-23T13:21:52.620 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-03-23T13:21:52.620 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-03-23T13:21:52.620 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-03-23T13:21:52.620 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 87, 2024-03-23T13:21:52.620 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-03-23T13:21:52.621 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 2, 2024-03-23T13:21:52.621 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 5, 2024-03-23T13:21:52.621 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 5, 2024-03-23T13:21:52.621 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-03-23T13:21:52.621 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-03-23T13:21:52.621 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 0, 2024-03-23T13:21:52.621 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 2, 2024-03-23T13:21:52.621 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 0 2024-03-23T13:21:52.621 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.622 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-03-23T13:21:52.622 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 0, 2024-03-23T13:21:52.622 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-03-23T13:21:52.622 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-03-23T13:21:52.622 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-03-23T13:21:52.622 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.622 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-03-23T13:21:52.622 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-03-23T13:21:52.622 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.623 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-03-23T13:21:52.623 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-03-23T13:21:52.623 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T13:21:52.623 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-03-23T13:21:52.623 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-03-23T13:21:52.623 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-03-23T13:21:52.623 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-03-23T13:21:52.623 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-03-23T13:21:52.623 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.624 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:21:52.624 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:52.624 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-03-23T13:21:52.624 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:52.624 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 769, 2024-03-23T13:21:52.624 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 4792117, 2024-03-23T13:21:52.624 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:52.624 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 769, 2024-03-23T13:21:52.625 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:52.625 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:52.625 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 769, 2024-03-23T13:21:52.625 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 4792117, 2024-03-23T13:21:52.625 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:52.625 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.625 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.625 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.626 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:52.626 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.626 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-03-23T13:21:52.626 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:52.626 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-03-23T13:21:52.626 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:52.626 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:52.626 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:52.626 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:52.627 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:52.627 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 674, 2024-03-23T13:21:52.627 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 4194309, 2024-03-23T13:21:52.627 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 674, 2024-03-23T13:21:52.627 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 4194309, 2024-03-23T13:21:52.627 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:52.627 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.627 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.628 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.628 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:52.628 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.628 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-03-23T13:21:52.628 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:52.628 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-03-23T13:21:52.628 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:52.629 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:52.629 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:52.629 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:52.629 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:52.629 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 674, 2024-03-23T13:21:52.629 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 674, 2024-03-23T13:21:52.629 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 674, 2024-03-23T13:21:52.629 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 674, 2024-03-23T13:21:52.629 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:52.630 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.630 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.630 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.630 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:52.630 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.630 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-03-23T13:21:52.630 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:52.630 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-03-23T13:21:52.631 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:52.631 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:52.631 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:52.631 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:52.631 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:52.631 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:52.631 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:52.631 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-03-23T13:21:52.632 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-03-23T13:21:52.632 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:52.632 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.632 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.632 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.632 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:52.632 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-03-23T13:21:52.632 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x558cc47066e0": { 2024-03-23T13:21:52.632 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-03-23T13:21:52.633 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-03-23T13:21:52.633 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-03-23T13:21:52.633 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-03-23T13:21:52.633 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-03-23T13:21:52.633 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:52.633 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:52.633 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-03-23T13:21:52.633 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-03-23T13:21:52.634 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-03-23T13:21:52.634 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-03-23T13:21:52.634 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-03-23T13:21:52.634 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-03-23T13:21:52.634 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-03-23T13:21:52.634 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-03-23T13:21:52.634 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:52.634 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-03-23T13:21:52.635 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-03-23T13:21:53.454 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-23T13:21:53.455 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-23T13:21:53.455 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-03-23T13:21:53.455 DEBUG:tasks.check_counter:Getting stats from c 2024-03-23T13:21:53.455 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-23T13:21:53.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:53 smithi007 ceph-mon[32184]: pgmap v412: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-03-23T13:21:53.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:53 smithi130 ceph-mon[36225]: pgmap v412: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-03-23T13:21:53.749 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:21:53.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:53 smithi155 ceph-mon[36157]: pgmap v412: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-03-23T13:21:55.447 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:55 smithi130 ceph-mon[36225]: pgmap v413: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-03-23T13:21:55.494 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:55 smithi155 ceph-mon[36157]: pgmap v413: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-03-23T13:21:55.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:55 smithi007 ceph-mon[32184]: pgmap v413: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-03-23T13:21:55.666 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-23T13:21:55.667 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:21:55.667 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2386, 2024-03-23T13:21:55.667 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 4658, 2024-03-23T13:21:55.667 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 784842, 2024-03-23T13:21:55.667 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 5121090, 2024-03-23T13:21:55.667 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 14, 2024-03-23T13:21:55.668 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-03-23T13:21:55.668 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.178993558, 2024-03-23T13:21:55.668 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.079349445, 2024-03-23T13:21:55.668 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.102580849, 2024-03-23T13:21:55.668 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.008671319, 2024-03-23T13:21:55.668 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:55.668 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4656, 2024-03-23T13:21:55.668 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.170026181, 2024-03-23T13:21:55.668 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000036517 2024-03-23T13:21:55.669 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.669 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:55.669 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5957, 2024-03-23T13:21:55.669 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001098102, 2024-03-23T13:21:55.669 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000184 2024-03-23T13:21:55.669 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.669 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-03-23T13:21:55.669 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 1440 2024-03-23T13:21:55.670 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.670 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:21:55.670 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 387, 2024-03-23T13:21:55.670 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 375, 2024-03-23T13:21:55.670 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 589765, 2024-03-23T13:21:55.670 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 366267, 2024-03-23T13:21:55.670 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 11, 2024-03-23T13:21:55.671 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-03-23T13:21:55.671 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.044165704, 2024-03-23T13:21:55.671 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.011919469, 2024-03-23T13:21:55.671 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.021144099, 2024-03-23T13:21:55.671 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.012264601, 2024-03-23T13:21:55.671 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:55.671 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 374, 2024-03-23T13:21:55.671 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.042591536, 2024-03-23T13:21:55.672 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000113881 2024-03-23T13:21:55.672 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.672 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:55.672 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 60, 2024-03-23T13:21:55.672 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000009441, 2024-03-23T13:21:55.672 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000157 2024-03-23T13:21:55.672 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.672 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 498720, 2024-03-23T13:21:55.672 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 52256 2024-03-23T13:21:55.673 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.673 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:21:55.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1981, 2024-03-23T13:21:55.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 3185, 2024-03-23T13:21:55.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1835932, 2024-03-23T13:21:55.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1198757, 2024-03-23T13:21:55.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 9, 2024-03-23T13:21:55.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-03-23T13:21:55.674 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.136810563, 2024-03-23T13:21:55.674 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.055271195, 2024-03-23T13:21:55.674 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.082633675, 2024-03-23T13:21:55.674 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.006595637, 2024-03-23T13:21:55.674 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:55.674 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3184, 2024-03-23T13:21:55.674 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.191128279, 2024-03-23T13:21:55.674 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000060027 2024-03-23T13:21:55.675 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:55.675 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4629, 2024-03-23T13:21:55.675 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000705259, 2024-03-23T13:21:55.675 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000152 2024-03-23T13:21:55.675 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-03-23T13:21:55.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 178288 2024-03-23T13:21:55.676 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.676 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-03-23T13:21:55.676 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-03-23T13:21:55.676 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-03-23T13:21:55.676 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.676 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-03-23T13:21:55.676 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-23T13:21:55.676 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-23T13:21:55.677 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 199, 2024-03-23T13:21:55.677 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.862653476, 2024-03-23T13:21:55.677 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004334942 2024-03-23T13:21:55.677 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:55.677 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.677 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-03-23T13:21:55.677 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-23T13:21:55.677 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-23T13:21:55.678 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 726, 2024-03-23T13:21:55.678 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.076042230, 2024-03-23T13:21:55.678 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000104741 2024-03-23T13:21:55.678 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:55.678 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.678 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-03-23T13:21:55.678 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 1252, 2024-03-23T13:21:55.678 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 1236, 2024-03-23T13:21:55.679 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-03-23T13:21:55.679 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1236, 2024-03-23T13:21:55.679 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.915990380, 2024-03-23T13:21:55.679 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001550154 2024-03-23T13:21:55.679 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.679 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-03-23T13:21:55.679 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 16, 2024-03-23T13:21:55.679 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-03-23T13:21:55.680 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-03-23T13:21:55.680 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 11, 2024-03-23T13:21:55.680 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-03-23T13:21:55.680 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-03-23T13:21:55.680 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 1162, 2024-03-23T13:21:55.680 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 42, 2024-03-23T13:21:55.680 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 1120, 2024-03-23T13:21:55.680 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-03-23T13:21:55.681 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 1083, 2024-03-23T13:21:55.681 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-03-23T13:21:55.681 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 31, 2024-03-23T13:21:55.681 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 31, 2024-03-23T13:21:55.681 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 121, 2024-03-23T13:21:55.681 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 2554, 2024-03-23T13:21:55.681 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 1773, 2024-03-23T13:21:55.681 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 14, 2024-03-23T13:21:55.682 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 26, 2024-03-23T13:21:55.682 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-03-23T13:21:55.682 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-03-23T13:21:55.682 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-03-23T13:21:55.682 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 3009, 2024-03-23T13:21:55.682 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-03-23T13:21:55.682 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-03-23T13:21:55.682 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-03-23T13:21:55.683 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 47, 2024-03-23T13:21:55.683 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 1136, 2024-03-23T13:21:55.683 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-03-23T13:21:55.683 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-03-23T13:21:55.683 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-03-23T13:21:55.683 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 3452, 2024-03-23T13:21:55.683 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 26211650, 2024-03-23T13:21:55.683 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-03-23T13:21:55.683 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T13:21:55.684 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-03-23T13:21:55.684 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T13:21:55.684 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-03-23T13:21:55.684 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-03-23T13:21:55.684 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T13:21:55.684 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T13:21:55.684 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-03-23T13:21:55.684 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-03-23T13:21:55.685 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 10, 2024-03-23T13:21:55.685 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 698, 2024-03-23T13:21:55.685 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T13:21:55.685 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T13:21:55.685 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T13:21:55.685 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 842, 2024-03-23T13:21:55.685 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 731, 2024-03-23T13:21:55.685 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 14, 2024-03-23T13:21:55.686 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 811 2024-03-23T13:21:55.686 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.686 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-03-23T13:21:55.686 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-03-23T13:21:55.686 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-03-23T13:21:55.686 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-03-23T13:21:55.686 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 250, 2024-03-23T13:21:55.686 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 2, 2024-03-23T13:21:55.687 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-03-23T13:21:55.687 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-03-23T13:21:55.687 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-03-23T13:21:55.687 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 424, 2024-03-23T13:21:55.687 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 424, 2024-03-23T13:21:55.687 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-03-23T13:21:55.687 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-03-23T13:21:55.687 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-03-23T13:21:55.687 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-03-23T13:21:55.688 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-03-23T13:21:55.688 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-03-23T13:21:55.688 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-03-23T13:21:55.688 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 0, 2024-03-23T13:21:55.688 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 0, 2024-03-23T13:21:55.688 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T13:21:55.688 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-03-23T13:21:55.688 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-03-23T13:21:55.688 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-03-23T13:21:55.689 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-03-23T13:21:55.689 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-03-23T13:21:55.689 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.689 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-03-23T13:21:55.689 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-03-23T13:21:55.689 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 2534, 2024-03-23T13:21:55.689 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-03-23T13:21:55.689 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-03-23T13:21:55.690 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 2534, 2024-03-23T13:21:55.690 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-03-23T13:21:55.690 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-03-23T13:21:55.690 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 5, 2024-03-23T13:21:55.690 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-03-23T13:21:55.690 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-03-23T13:21:55.690 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 5, 2024-03-23T13:21:55.690 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 2, 2024-03-23T13:21:55.691 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-03-23T13:21:55.691 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-03-23T13:21:55.691 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-03-23T13:21:55.691 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 8583032, 2024-03-23T13:21:55.691 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-03-23T13:21:55.691 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-03-23T13:21:55.691 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 71, 2024-03-23T13:21:55.691 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.181800531, 2024-03-23T13:21:55.692 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002560570 2024-03-23T13:21:55.692 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.692 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-03-23T13:21:55.692 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.692 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-03-23T13:21:55.692 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 864, 2024-03-23T13:21:55.692 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 1288, 2024-03-23T13:21:55.692 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 424, 2024-03-23T13:21:55.693 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 186, 2024-03-23T13:21:55.693 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 187, 2024-03-23T13:21:55.693 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 1, 2024-03-23T13:21:55.693 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 1162, 2024-03-23T13:21:55.693 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 1586, 2024-03-23T13:21:55.693 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 424, 2024-03-23T13:21:55.693 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 31, 2024-03-23T13:21:55.693 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 1268, 2024-03-23T13:21:55.694 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 1237, 2024-03-23T13:21:55.694 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 62956, 2024-03-23T13:21:55.694 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 223612 2024-03-23T13:21:55.694 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.694 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-03-23T13:21:55.694 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 2065, 2024-03-23T13:21:55.694 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-03-23T13:21:55.695 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 1252, 2024-03-23T13:21:55.695 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 6, 2024-03-23T13:21:55.695 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 688, 2024-03-23T13:21:55.695 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-03-23T13:21:55.695 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 263, 2024-03-23T13:21:55.695 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.398990579, 2024-03-23T13:21:55.695 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001517074 2024-03-23T13:21:55.695 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.696 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-03-23T13:21:55.696 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-03-23T13:21:55.696 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001231683, 2024-03-23T13:21:55.696 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000615841 2024-03-23T13:21:55.696 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.696 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-03-23T13:21:55.696 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.696 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.697 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.697 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.697 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-03-23T13:21:55.697 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.697 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.697 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.697 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.697 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-03-23T13:21:55.698 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 545, 2024-03-23T13:21:55.698 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.280278426, 2024-03-23T13:21:55.698 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000514272 2024-03-23T13:21:55.698 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.698 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-03-23T13:21:55.698 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.698 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.698 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.699 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.699 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-03-23T13:21:55.699 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.699 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.699 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.699 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.699 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-03-23T13:21:55.699 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.699 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.700 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.700 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.700 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-03-23T13:21:55.700 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.700 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.700 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.700 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.700 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-03-23T13:21:55.700 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.701 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.701 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.701 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.701 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-03-23T13:21:55.701 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.702 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.702 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.702 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.702 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-03-23T13:21:55.702 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.703 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.703 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.703 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.703 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-03-23T13:21:55.703 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.703 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.703 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.703 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.704 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-03-23T13:21:55.704 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.704 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.704 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.704 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.704 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-03-23T13:21:55.704 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-03-23T13:21:55.704 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000257472, 2024-03-23T13:21:55.704 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000257472 2024-03-23T13:21:55.705 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.705 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-03-23T13:21:55.705 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-03-23T13:21:55.705 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001180527, 2024-03-23T13:21:55.705 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001180527 2024-03-23T13:21:55.705 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.705 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-03-23T13:21:55.705 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.706 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.706 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.706 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.706 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-03-23T13:21:55.706 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.706 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.706 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.706 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.707 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-03-23T13:21:55.707 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.707 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.707 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.707 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.707 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-03-23T13:21:55.707 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.707 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.707 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.708 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.708 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-03-23T13:21:55.708 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.708 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.708 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.708 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.708 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-03-23T13:21:55.708 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.708 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.709 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.709 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.709 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-03-23T13:21:55.709 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.709 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.709 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.709 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.709 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-03-23T13:21:55.709 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.710 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.710 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.710 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.710 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-03-23T13:21:55.710 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.710 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.710 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.710 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.711 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-03-23T13:21:55.711 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.711 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.711 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.711 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.711 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-03-23T13:21:55.711 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.711 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.711 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.712 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.712 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-03-23T13:21:55.712 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.712 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.712 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.712 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.712 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-03-23T13:21:55.712 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 424, 2024-03-23T13:21:55.713 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.234051693, 2024-03-23T13:21:55.713 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002910499 2024-03-23T13:21:55.713 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.713 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-03-23T13:21:55.713 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-03-23T13:21:55.713 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-03-23T13:21:55.713 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.713 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.714 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.714 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:55.714 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.714 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-03-23T13:21:55.714 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-03-23T13:21:55.714 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 2, 2024-03-23T13:21:55.714 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 1, 2024-03-23T13:21:55.714 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2024-03-23T13:21:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-03-23T13:21:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 1117, 2024-03-23T13:21:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 1117, 2024-03-23T13:21:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 133, 2024-03-23T13:21:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-03-23T13:21:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-03-23T13:21:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:21:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-03-23T13:21:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:21:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:21:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:21:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:21:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:21:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:21:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:21:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:21:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:21:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:21:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:21:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:21:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:21:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-03-23T13:21:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:21:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-03-23T13:21:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:21:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:21:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:21:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:21:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:21:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:21:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:21:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-03-23T13:21:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:21:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:21:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:21:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-03-23T13:21:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-03-23T13:21:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-03-23T13:21:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:21:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:21:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:21:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:21:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 1244514, 2024-03-23T13:21:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 1204, 2024-03-23T13:21:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 880, 2024-03-23T13:21:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 10, 2024-03-23T13:21:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-03-23T13:21:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-03-23T13:21:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:21:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-03-23T13:21:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:21:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-03-23T13:21:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 7528, 2024-03-23T13:21:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 158, 2024-03-23T13:21:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:21:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:21:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-03-23T13:21:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-03-23T13:21:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 3305394, 2024-03-23T13:21:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 36376, 2024-03-23T13:21:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-03-23T13:21:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-03-23T13:21:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-03-23T13:21:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-03-23T13:21:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-03-23T13:21:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-03-23T13:21:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-03-23T13:21:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 603, 2024-03-23T13:21:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 4532813, 2024-03-23T13:21:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 33, 2024-03-23T13:21:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 603, 2024-03-23T13:21:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-03-23T13:21:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 603, 2024-03-23T13:21:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 2.387949610, 2024-03-23T13:21:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003960115 2024-03-23T13:21:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-03-23T13:21:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-03-23T13:21:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 603, 2024-03-23T13:21:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 657 2024-03-23T13:21:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 603, 2024-03-23T13:21:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 14, 2024-03-23T13:21:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 589, 2024-03-23T13:21:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-03-23T13:21:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-03-23T13:21:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2024-03-23T13:21:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 10, 2024-03-23T13:21:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 14, 2024-03-23T13:21:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 85, 2024-03-23T13:21:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 18, 2024-03-23T13:21:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-03-23T13:21:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-03-23T13:21:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-03-23T13:21:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-03-23T13:21:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 437, 2024-03-23T13:21:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-03-23T13:21:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-03-23T13:21:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-03-23T13:21:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 0, 2024-03-23T13:21:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 20, 2024-03-23T13:21:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-03-23T13:21:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-03-23T13:21:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-03-23T13:21:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-03-23T13:21:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-03-23T13:21:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-03-23T13:21:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T13:21:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-03-23T13:21:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-03-23T13:21:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-03-23T13:21:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-03-23T13:21:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-03-23T13:21:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-03-23T13:21:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-03-23T13:21:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-03-23T13:21:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-03-23T13:21:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-03-23T13:21:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-03-23T13:21:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-03-23T13:21:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-03-23T13:21:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-03-23T13:21:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-03-23T13:21:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-03-23T13:21:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-03-23T13:21:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-03-23T13:21:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-03-23T13:21:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 87, 2024-03-23T13:21:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-03-23T13:21:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 2, 2024-03-23T13:21:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2024-03-23T13:21:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2024-03-23T13:21:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-03-23T13:21:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-03-23T13:21:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 63, 2024-03-23T13:21:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 0, 2024-03-23T13:21:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 10 2024-03-23T13:21:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-03-23T13:21:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-03-23T13:21:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-03-23T13:21:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-03-23T13:21:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-03-23T13:21:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-03-23T13:21:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-03-23T13:21:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-03-23T13:21:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-03-23T13:21:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 6, 2024-03-23T13:21:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-03-23T13:21:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 6, 2024-03-23T13:21:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 424, 2024-03-23T13:21:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 424, 2024-03-23T13:21:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-03-23T13:21:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:21:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:21:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-23T13:21:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:21:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 4750, 2024-03-23T13:21:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2773520, 2024-03-23T13:21:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 4750, 2024-03-23T13:21:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:21:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:21:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 4750, 2024-03-23T13:21:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2773520, 2024-03-23T13:21:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:21:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-03-23T13:21:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:21:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-23T13:21:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:21:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-23T13:21:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-23T13:21:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 603, 2024-03-23T13:21:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 4575637, 2024-03-23T13:21:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 603, 2024-03-23T13:21:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 4575637, 2024-03-23T13:21:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:21:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-03-23T13:21:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:21:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-03-23T13:21:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:21:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-23T13:21:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-23T13:21:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 603, 2024-03-23T13:21:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 603, 2024-03-23T13:21:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 603, 2024-03-23T13:21:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 603, 2024-03-23T13:21:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:21:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-03-23T13:21:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:21:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-23T13:21:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:21:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 424, 2024-03-23T13:21:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 42824, 2024-03-23T13:21:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 424, 2024-03-23T13:21:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:21:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:21:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 14, 2024-03-23T13:21:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 42824, 2024-03-23T13:21:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:21:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55f14e9de500": { 2024-03-23T13:21:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:21:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-23T13:21:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:21:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 2534, 2024-03-23T13:21:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 4388728, 2024-03-23T13:21:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 2534, 2024-03-23T13:21:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 71, 2024-03-23T13:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 4388728, 2024-03-23T13:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-23T13:21:56.477 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-23T13:21:56.477 INFO:tasks.check_counter:Daemon mds.c mds.imported=47 2024-03-23T13:21:56.477 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-03-23T13:21:56.477 DEBUG:tasks.check_counter:Getting stats from f 2024-03-23T13:21:56.477 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-23T13:21:56.805 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:21:57.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:57 smithi130 ceph-mon[36225]: pgmap v414: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-03-23T13:21:57.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:57 smithi155 ceph-mon[36157]: pgmap v414: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-03-23T13:21:57.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:57 smithi007 ceph-mon[32184]: pgmap v414: 97 pgs: 97 active+clean; 46 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-03-23T13:21:58.685 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-23T13:21:58.686 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:21:58.686 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2957, 2024-03-23T13:21:58.686 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 6325, 2024-03-23T13:21:58.686 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1847420, 2024-03-23T13:21:58.686 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 5750533, 2024-03-23T13:21:58.686 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 10, 2024-03-23T13:21:58.687 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-03-23T13:21:58.687 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.218514854, 2024-03-23T13:21:58.687 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.095905122, 2024-03-23T13:21:58.687 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.116124805, 2024-03-23T13:21:58.687 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.008475954, 2024-03-23T13:21:58.687 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:58.687 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6325, 2024-03-23T13:21:58.688 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.260502266, 2024-03-23T13:21:58.688 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000041186 2024-03-23T13:21:58.688 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.688 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:58.688 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 8389, 2024-03-23T13:21:58.688 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001549657, 2024-03-23T13:21:58.688 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000184 2024-03-23T13:21:58.688 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.688 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T13:21:58.689 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T13:21:58.689 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.689 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:21:58.689 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2004, 2024-03-23T13:21:58.689 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 4457, 2024-03-23T13:21:58.689 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1081878, 2024-03-23T13:21:58.689 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 2264258, 2024-03-23T13:21:58.689 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 14, 2024-03-23T13:21:58.690 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 9, 2024-03-23T13:21:58.690 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.149897016, 2024-03-23T13:21:58.690 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.062239101, 2024-03-23T13:21:58.690 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.095555166, 2024-03-23T13:21:58.690 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.005663354, 2024-03-23T13:21:58.690 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:58.690 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4454, 2024-03-23T13:21:58.691 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.106410623, 2024-03-23T13:21:58.691 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000023891 2024-03-23T13:21:58.691 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.691 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:58.691 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5179, 2024-03-23T13:21:58.691 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000893792, 2024-03-23T13:21:58.691 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000172 2024-03-23T13:21:58.691 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.692 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 487376, 2024-03-23T13:21:58.692 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 52896 2024-03-23T13:21:58.692 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.692 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:21:58.692 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 227, 2024-03-23T13:21:58.692 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 281, 2024-03-23T13:21:58.692 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 85082, 2024-03-23T13:21:58.692 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 346798, 2024-03-23T13:21:58.692 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 9, 2024-03-23T13:21:58.693 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 7, 2024-03-23T13:21:58.693 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.022844126, 2024-03-23T13:21:58.693 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.009084531, 2024-03-23T13:21:58.693 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.008269816, 2024-03-23T13:21:58.693 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.004297066, 2024-03-23T13:21:58.693 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:21:58.693 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 280, 2024-03-23T13:21:58.693 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.033668817, 2024-03-23T13:21:58.694 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000120245 2024-03-23T13:21:58.694 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.694 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:21:58.694 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 98, 2024-03-23T13:21:58.695 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000016125, 2024-03-23T13:21:58.695 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000164 2024-03-23T13:21:58.695 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.695 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-03-23T13:21:58.695 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 178464 2024-03-23T13:21:58.696 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.696 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-03-23T13:21:58.696 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-03-23T13:21:58.697 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-03-23T13:21:58.697 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.697 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-03-23T13:21:58.697 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-23T13:21:58.697 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-23T13:21:58.697 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 313, 2024-03-23T13:21:58.697 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.766089950, 2024-03-23T13:21:58.698 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002447571 2024-03-23T13:21:58.698 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:58.698 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.698 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-03-23T13:21:58.698 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-23T13:21:58.698 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-23T13:21:58.698 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 683, 2024-03-23T13:21:58.698 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.071751508, 2024-03-23T13:21:58.699 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000105053 2024-03-23T13:21:58.699 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:58.699 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.699 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-03-23T13:21:58.699 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 1260, 2024-03-23T13:21:58.699 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 1260, 2024-03-23T13:21:58.699 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-03-23T13:21:58.699 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1260, 2024-03-23T13:21:58.699 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 9.053869250, 2024-03-23T13:21:58.700 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.007185610 2024-03-23T13:21:58.700 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.700 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-03-23T13:21:58.700 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-03-23T13:21:58.700 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-03-23T13:21:58.700 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-03-23T13:21:58.700 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 11, 2024-03-23T13:21:58.700 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-03-23T13:21:58.701 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-03-23T13:21:58.701 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 1201, 2024-03-23T13:21:58.701 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 105, 2024-03-23T13:21:58.701 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 1096, 2024-03-23T13:21:58.701 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-03-23T13:21:58.701 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 998, 2024-03-23T13:21:58.701 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 4, 2024-03-23T13:21:58.701 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 177, 2024-03-23T13:21:58.702 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 177, 2024-03-23T13:21:58.702 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 61, 2024-03-23T13:21:58.702 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 2896, 2024-03-23T13:21:58.702 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 1307, 2024-03-23T13:21:58.702 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-03-23T13:21:58.702 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 37, 2024-03-23T13:21:58.702 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-03-23T13:21:58.703 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-03-23T13:21:58.703 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 3, 2024-03-23T13:21:58.703 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 14755, 2024-03-23T13:21:58.703 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 1, 2024-03-23T13:21:58.703 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-03-23T13:21:58.703 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-03-23T13:21:58.703 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 52, 2024-03-23T13:21:58.703 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 946, 2024-03-23T13:21:58.703 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-03-23T13:21:58.704 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-03-23T13:21:58.704 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-03-23T13:21:58.704 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 3452, 2024-03-23T13:21:58.704 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 26211650, 2024-03-23T13:21:58.704 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-03-23T13:21:58.704 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T13:21:58.704 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-03-23T13:21:58.704 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T13:21:58.704 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-03-23T13:21:58.705 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-03-23T13:21:58.705 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T13:21:58.705 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T13:21:58.705 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-03-23T13:21:58.705 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-03-23T13:21:58.705 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 22, 2024-03-23T13:21:58.705 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 830, 2024-03-23T13:21:58.705 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 2, 2024-03-23T13:21:58.705 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T13:21:58.706 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T13:21:58.706 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 395, 2024-03-23T13:21:58.706 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 370, 2024-03-23T13:21:58.706 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 14, 2024-03-23T13:21:58.706 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 627 2024-03-23T13:21:58.706 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.706 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-03-23T13:21:58.706 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-03-23T13:21:58.707 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-03-23T13:21:58.707 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-03-23T13:21:58.707 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 47, 2024-03-23T13:21:58.707 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 39, 2024-03-23T13:21:58.707 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-03-23T13:21:58.707 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-03-23T13:21:58.707 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-03-23T13:21:58.708 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 393, 2024-03-23T13:21:58.708 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 393, 2024-03-23T13:21:58.708 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-03-23T13:21:58.708 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-03-23T13:21:58.708 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-03-23T13:21:58.708 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-03-23T13:21:58.708 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-03-23T13:21:58.708 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-03-23T13:21:58.708 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-03-23T13:21:58.709 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 0, 2024-03-23T13:21:58.709 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 0, 2024-03-23T13:21:58.709 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T13:21:58.709 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-03-23T13:21:58.709 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-03-23T13:21:58.709 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-03-23T13:21:58.709 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-03-23T13:21:58.709 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-03-23T13:21:58.709 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.710 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-03-23T13:21:58.710 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-03-23T13:21:58.710 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 2355, 2024-03-23T13:21:58.710 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-03-23T13:21:58.710 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-03-23T13:21:58.710 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 2355, 2024-03-23T13:21:58.710 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-03-23T13:21:58.710 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-03-23T13:21:58.711 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 4, 2024-03-23T13:21:58.711 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-03-23T13:21:58.711 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-03-23T13:21:58.711 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 4, 2024-03-23T13:21:58.711 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 2, 2024-03-23T13:21:58.711 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-03-23T13:21:58.711 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-03-23T13:21:58.711 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-03-23T13:21:58.711 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 7942341, 2024-03-23T13:21:58.712 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-03-23T13:21:58.712 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-03-23T13:21:58.712 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 133, 2024-03-23T13:21:58.712 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.268488919, 2024-03-23T13:21:58.712 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002018713 2024-03-23T13:21:58.712 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.712 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-03-23T13:21:58.712 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.712 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-03-23T13:21:58.713 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 882, 2024-03-23T13:21:58.713 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 1276, 2024-03-23T13:21:58.713 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 394, 2024-03-23T13:21:58.713 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 87, 2024-03-23T13:21:58.713 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 89, 2024-03-23T13:21:58.713 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 2, 2024-03-23T13:21:58.713 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 1201, 2024-03-23T13:21:58.713 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 1616, 2024-03-23T13:21:58.713 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 415, 2024-03-23T13:21:58.714 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 170, 2024-03-23T13:21:58.714 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 1242, 2024-03-23T13:21:58.714 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 1072, 2024-03-23T13:21:58.714 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 62860, 2024-03-23T13:21:58.714 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 223612 2024-03-23T13:21:58.714 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.714 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-03-23T13:21:58.714 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 2964, 2024-03-23T13:21:58.715 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-03-23T13:21:58.715 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 1260, 2024-03-23T13:21:58.715 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 2, 2024-03-23T13:21:58.715 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 774, 2024-03-23T13:21:58.715 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-03-23T13:21:58.715 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 369, 2024-03-23T13:21:58.715 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.611647378, 2024-03-23T13:21:58.715 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001657580 2024-03-23T13:21:58.715 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.715 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-03-23T13:21:58.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.716 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.716 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.716 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-03-23T13:21:58.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.716 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.716 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.717 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-03-23T13:21:58.717 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.717 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.717 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.717 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.717 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-03-23T13:21:58.717 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 370, 2024-03-23T13:21:58.717 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.128420405, 2024-03-23T13:21:58.717 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000347082 2024-03-23T13:21:58.718 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.718 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-03-23T13:21:58.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.718 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.718 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.718 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-03-23T13:21:58.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.719 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.719 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.719 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.719 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-03-23T13:21:58.719 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.719 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.719 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.719 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.719 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-03-23T13:21:58.720 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.720 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.720 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.720 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.720 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-03-23T13:21:58.720 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.720 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.720 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.720 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.721 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-03-23T13:21:58.721 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.721 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.722 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.722 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.722 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-03-23T13:21:58.722 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3, 2024-03-23T13:21:58.722 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.006748507, 2024-03-23T13:21:58.722 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002249502 2024-03-23T13:21:58.722 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.723 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-03-23T13:21:58.723 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.723 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.723 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.723 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.723 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-03-23T13:21:58.723 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.723 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.724 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.724 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.724 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-03-23T13:21:58.724 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 86, 2024-03-23T13:21:58.724 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.022228633, 2024-03-23T13:21:58.724 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000258472 2024-03-23T13:21:58.724 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.725 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-03-23T13:21:58.725 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 21, 2024-03-23T13:21:58.725 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.016765682, 2024-03-23T13:21:58.725 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000798365 2024-03-23T13:21:58.725 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.725 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-03-23T13:21:58.725 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 15, 2024-03-23T13:21:58.726 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 7.097630787, 2024-03-23T13:21:58.726 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.473175385 2024-03-23T13:21:58.726 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.726 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-03-23T13:21:58.726 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.726 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.726 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.727 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.727 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-03-23T13:21:58.727 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.727 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.727 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.727 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.728 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-03-23T13:21:58.728 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.728 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.728 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.728 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.728 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-03-23T13:21:58.728 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.728 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.729 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.729 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.729 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-03-23T13:21:58.729 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.729 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.729 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.729 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.730 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-03-23T13:21:58.730 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-03-23T13:21:58.730 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.008380218, 2024-03-23T13:21:58.730 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004190109 2024-03-23T13:21:58.730 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.730 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-03-23T13:21:58.730 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.731 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.731 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.731 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.731 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-03-23T13:21:58.731 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.731 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.732 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.732 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.732 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-03-23T13:21:58.732 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.732 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.732 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.732 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.733 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-03-23T13:21:58.733 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.733 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.733 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.733 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.733 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-03-23T13:21:58.734 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-03-23T13:21:58.734 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001006379, 2024-03-23T13:21:58.734 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001006379 2024-03-23T13:21:58.734 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.734 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-03-23T13:21:58.734 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 393, 2024-03-23T13:21:58.734 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.161041261, 2024-03-23T13:21:58.735 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002954303 2024-03-23T13:21:58.735 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.735 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-03-23T13:21:58.735 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-03-23T13:21:58.735 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-03-23T13:21:58.735 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.736 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.736 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.736 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:58.736 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.736 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-03-23T13:21:58.736 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-03-23T13:21:58.736 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 1, 2024-03-23T13:21:58.736 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 0, 2024-03-23T13:21:58.737 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2024-03-23T13:21:58.737 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-03-23T13:21:58.737 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 1036, 2024-03-23T13:21:58.737 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 1036, 2024-03-23T13:21:58.737 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1487, 2024-03-23T13:21:58.737 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-03-23T13:21:58.737 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.737 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-03-23T13:21:58.738 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:21:58.738 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-03-23T13:21:58.738 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:21:58.738 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:21:58.738 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:21:58.738 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:21:58.738 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:21:58.739 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:21:58.739 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:21:58.739 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:21:58.739 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:21:58.739 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:21:58.739 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:21:58.739 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:21:58.740 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:21:58.740 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-03-23T13:21:58.740 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:21:58.740 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-03-23T13:21:58.740 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:21:58.740 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:21:58.740 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:21:58.740 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:21:58.741 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:21:58.741 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:21:58.741 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:21:58.741 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-03-23T13:21:58.741 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:21:58.741 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:21:58.741 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:21:58.742 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-03-23T13:21:58.742 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-03-23T13:21:58.742 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-03-23T13:21:58.742 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:21:58.742 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:21:58.742 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:21:58.742 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:21:58.742 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 1008804, 2024-03-23T13:21:58.743 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 1167, 2024-03-23T13:21:58.743 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 792, 2024-03-23T13:21:58.743 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 9, 2024-03-23T13:21:58.743 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-03-23T13:21:58.743 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-03-23T13:21:58.743 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:21:58.743 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-03-23T13:21:58.743 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:21:58.744 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-03-23T13:21:58.744 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 7528, 2024-03-23T13:21:58.744 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 158, 2024-03-23T13:21:58.744 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:21:58.744 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:21:58.744 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-03-23T13:21:58.744 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-03-23T13:21:58.745 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 3204438, 2024-03-23T13:21:58.745 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 35516, 2024-03-23T13:21:58.745 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-03-23T13:21:58.745 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-03-23T13:21:58.746 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-03-23T13:21:58.746 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-03-23T13:21:58.746 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.746 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-03-23T13:21:58.746 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-03-23T13:21:58.746 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-03-23T13:21:58.747 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 635, 2024-03-23T13:21:58.747 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 3849608, 2024-03-23T13:21:58.747 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 33, 2024-03-23T13:21:58.747 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 635, 2024-03-23T13:21:58.747 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-03-23T13:21:58.747 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 635, 2024-03-23T13:21:58.747 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.940959023, 2024-03-23T13:21:58.748 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003056628 2024-03-23T13:21:58.748 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.748 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-03-23T13:21:58.748 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-03-23T13:21:58.748 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 635, 2024-03-23T13:21:58.748 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 688 2024-03-23T13:21:58.748 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 635, 2024-03-23T13:21:58.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 14, 2024-03-23T13:21:58.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 621, 2024-03-23T13:21:58.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-03-23T13:21:58.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-03-23T13:21:58.749 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2024-03-23T13:21:58.749 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 10, 2024-03-23T13:21:58.749 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 14, 2024-03-23T13:21:58.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 147, 2024-03-23T13:21:58.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 16, 2024-03-23T13:21:58.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-03-23T13:21:58.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-03-23T13:21:58.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-03-23T13:21:58.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-03-23T13:21:58.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 405, 2024-03-23T13:21:58.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-03-23T13:21:58.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-03-23T13:21:58.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-03-23T13:21:58.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 0, 2024-03-23T13:21:58.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 20, 2024-03-23T13:21:58.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-03-23T13:21:58.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-03-23T13:21:58.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-03-23T13:21:58.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-03-23T13:21:58.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-03-23T13:21:58.752 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-03-23T13:21:58.752 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T13:21:58.752 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-03-23T13:21:58.752 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-03-23T13:21:58.752 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 2, 2024-03-23T13:21:58.752 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-03-23T13:21:58.752 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-03-23T13:21:58.752 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-03-23T13:21:58.753 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-03-23T13:21:58.753 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-03-23T13:21:58.753 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-03-23T13:21:58.753 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-03-23T13:21:58.753 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-03-23T13:21:58.753 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-03-23T13:21:58.753 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-03-23T13:21:58.753 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-03-23T13:21:58.753 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-03-23T13:21:58.754 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-03-23T13:21:58.754 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-03-23T13:21:58.754 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-03-23T13:21:58.754 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-03-23T13:21:58.754 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 87, 2024-03-23T13:21:58.754 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-03-23T13:21:58.754 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 2, 2024-03-23T13:21:58.754 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2024-03-23T13:21:58.755 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2024-03-23T13:21:58.755 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-03-23T13:21:58.755 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-03-23T13:21:58.755 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 65, 2024-03-23T13:21:58.755 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 0, 2024-03-23T13:21:58.755 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 9 2024-03-23T13:21:58.755 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.755 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-03-23T13:21:58.756 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-03-23T13:21:58.756 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-03-23T13:21:58.756 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-03-23T13:21:58.756 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-03-23T13:21:58.756 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.756 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-03-23T13:21:58.756 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-03-23T13:21:58.756 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.756 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-03-23T13:21:58.757 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-03-23T13:21:58.757 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 6, 2024-03-23T13:21:58.757 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-03-23T13:21:58.757 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 6, 2024-03-23T13:21:58.757 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 393, 2024-03-23T13:21:58.757 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 393, 2024-03-23T13:21:58.757 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-03-23T13:21:58.757 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.757 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:21:58.758 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:21:58.758 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-23T13:21:58.758 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:21:58.758 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 5184, 2024-03-23T13:21:58.758 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2538692, 2024-03-23T13:21:58.758 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:58.758 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 5184, 2024-03-23T13:21:58.758 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:21:58.759 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:21:58.759 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 5184, 2024-03-23T13:21:58.759 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2538692, 2024-03-23T13:21:58.759 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:21:58.759 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.759 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.759 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.760 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:58.760 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.760 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-03-23T13:21:58.760 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:21:58.760 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-23T13:21:58.760 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:21:58.760 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-23T13:21:58.761 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-23T13:21:58.761 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:58.761 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:58.761 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 635, 2024-03-23T13:21:58.761 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 3889301, 2024-03-23T13:21:58.761 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 635, 2024-03-23T13:21:58.761 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3889301, 2024-03-23T13:21:58.762 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:21:58.762 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.762 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.762 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.762 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:58.762 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.762 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-03-23T13:21:58.762 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:21:58.763 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-03-23T13:21:58.763 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:21:58.763 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-23T13:21:58.763 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-23T13:21:58.763 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:58.763 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-23T13:21:58.763 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 635, 2024-03-23T13:21:58.763 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 635, 2024-03-23T13:21:58.764 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 635, 2024-03-23T13:21:58.764 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 635, 2024-03-23T13:21:58.764 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:21:58.764 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.764 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.764 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.764 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:58.764 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.765 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-03-23T13:21:58.765 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:21:58.765 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-23T13:21:58.765 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:21:58.765 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 393, 2024-03-23T13:21:58.765 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 39693, 2024-03-23T13:21:58.765 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:58.765 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 393, 2024-03-23T13:21:58.765 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:21:58.766 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:21:58.766 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 14, 2024-03-23T13:21:58.766 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 39693, 2024-03-23T13:21:58.766 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:21:58.766 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.766 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.766 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.766 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:58.767 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:21:58.767 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55ee0e7ba500": { 2024-03-23T13:21:58.767 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 18244, 2024-03-23T13:21:58.767 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-23T13:21:58.767 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:21:58.767 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 2355, 2024-03-23T13:21:58.767 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 3748037, 2024-03-23T13:21:58.767 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:21:58.767 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 2355, 2024-03-23T13:21:58.768 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:21:58.768 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:21:58.768 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 133, 2024-03-23T13:21:58.768 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3729793, 2024-03-23T13:21:58.768 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:21:58.768 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:21:58.768 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:21:58.768 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:21:58.768 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:58.769 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:21:58.769 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-23T13:21:59.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:21:59 smithi155 ceph-mon[36157]: pgmap v415: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 16 op/s 2024-03-23T13:21:59.579 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-23T13:21:59.580 INFO:tasks.check_counter:Daemon mds.f mds.imported=52 2024-03-23T13:21:59.580 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-03-23T13:21:59.580 DEBUG:tasks.check_counter:Getting stats from i 2024-03-23T13:21:59.580 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-23T13:21:59.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:21:59 smithi007 ceph-mon[32184]: pgmap v415: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 16 op/s 2024-03-23T13:21:59.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:21:59 smithi130 ceph-mon[36225]: pgmap v415: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 16 op/s 2024-03-23T13:21:59.893 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:22:01.642 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:01 smithi155 ceph-mon[36157]: pgmap v416: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 64 KiB/s wr, 12 op/s 2024-03-23T13:22:01.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:01 smithi130 ceph-mon[36225]: pgmap v416: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 64 KiB/s wr, 12 op/s 2024-03-23T13:22:01.777 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-23T13:22:01.777 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:22:01.778 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1872, 2024-03-23T13:22:01.778 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2960, 2024-03-23T13:22:01.778 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1731312, 2024-03-23T13:22:01.778 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 5195126, 2024-03-23T13:22:01.778 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 10, 2024-03-23T13:22:01.778 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-03-23T13:22:01.778 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.153568056, 2024-03-23T13:22:01.779 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.065227944, 2024-03-23T13:22:01.779 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.080136351, 2024-03-23T13:22:01.779 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.010190949, 2024-03-23T13:22:01.779 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:22:01.779 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2962, 2024-03-23T13:22:01.779 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.143540185, 2024-03-23T13:22:01.779 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000048460 2024-03-23T13:22:01.779 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.779 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:22:01.780 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4268, 2024-03-23T13:22:01.780 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000681559, 2024-03-23T13:22:01.780 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000159 2024-03-23T13:22:01.780 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.780 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T13:22:01.780 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T13:22:01.780 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.780 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:22:01.781 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2620, 2024-03-23T13:22:01.781 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 5111, 2024-03-23T13:22:01.781 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 828176, 2024-03-23T13:22:01.781 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 2767717, 2024-03-23T13:22:01.781 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 15, 2024-03-23T13:22:01.781 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 7, 2024-03-23T13:22:01.781 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.188928169, 2024-03-23T13:22:01.781 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.082921182, 2024-03-23T13:22:01.782 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.120942310, 2024-03-23T13:22:01.782 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.006025894, 2024-03-23T13:22:01.782 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:22:01.782 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5105, 2024-03-23T13:22:01.782 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.321767964, 2024-03-23T13:22:01.782 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000063029 2024-03-23T13:22:01.782 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.782 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:22:01.782 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6457, 2024-03-23T13:22:01.783 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001199267, 2024-03-23T13:22:01.783 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000185 2024-03-23T13:22:01.783 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.783 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-03-23T13:22:01.783 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 180192 2024-03-23T13:22:01.783 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.783 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:22:01.783 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 351, 2024-03-23T13:22:01.784 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 334, 2024-03-23T13:22:01.784 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 635179, 2024-03-23T13:22:01.784 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 316106, 2024-03-23T13:22:01.784 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 10, 2024-03-23T13:22:01.784 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-03-23T13:22:01.784 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.038363384, 2024-03-23T13:22:01.784 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.011134315, 2024-03-23T13:22:01.784 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.020080314, 2024-03-23T13:22:01.785 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.009343442, 2024-03-23T13:22:01.785 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:22:01.785 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 334, 2024-03-23T13:22:01.785 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.038883657, 2024-03-23T13:22:01.785 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000116418 2024-03-23T13:22:01.785 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.785 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:22:01.785 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 69, 2024-03-23T13:22:01.785 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000009852, 2024-03-23T13:22:01.786 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000142 2024-03-23T13:22:01.786 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.786 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 505312, 2024-03-23T13:22:01.786 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 52704 2024-03-23T13:22:01.786 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.786 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-03-23T13:22:01.786 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-03-23T13:22:01.786 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-03-23T13:22:01.787 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.787 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-03-23T13:22:01.787 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-23T13:22:01.787 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-23T13:22:01.787 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 204, 2024-03-23T13:22:01.787 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.910716644, 2024-03-23T13:22:01.787 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004464297 2024-03-23T13:22:01.787 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:01.787 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.788 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-03-23T13:22:01.788 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-23T13:22:01.788 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-23T13:22:01.788 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 756, 2024-03-23T13:22:01.788 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.086781729, 2024-03-23T13:22:01.788 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000114790 2024-03-23T13:22:01.788 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:01.788 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.788 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-03-23T13:22:01.789 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 1461, 2024-03-23T13:22:01.789 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 1448, 2024-03-23T13:22:01.789 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-03-23T13:22:01.789 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1448, 2024-03-23T13:22:01.789 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 2.132317137, 2024-03-23T13:22:01.789 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001472594 2024-03-23T13:22:01.790 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.790 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-03-23T13:22:01.790 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 13, 2024-03-23T13:22:01.790 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-03-23T13:22:01.790 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-03-23T13:22:01.790 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 11, 2024-03-23T13:22:01.790 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-03-23T13:22:01.790 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-03-23T13:22:01.791 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 1413, 2024-03-23T13:22:01.791 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 188, 2024-03-23T13:22:01.791 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 1225, 2024-03-23T13:22:01.791 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-03-23T13:22:01.791 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 1094, 2024-03-23T13:22:01.791 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-03-23T13:22:01.791 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 268, 2024-03-23T13:22:01.791 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 268, 2024-03-23T13:22:01.792 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 32, 2024-03-23T13:22:01.792 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 2933, 2024-03-23T13:22:01.792 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 1983, 2024-03-23T13:22:01.792 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 13, 2024-03-23T13:22:01.792 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 13, 2024-03-23T13:22:01.792 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-03-23T13:22:01.792 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-03-23T13:22:01.792 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-03-23T13:22:01.793 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 8957, 2024-03-23T13:22:01.793 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 1, 2024-03-23T13:22:01.793 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-03-23T13:22:01.793 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-03-23T13:22:01.793 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 28, 2024-03-23T13:22:01.793 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 1347, 2024-03-23T13:22:01.793 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-03-23T13:22:01.793 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-03-23T13:22:01.794 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-03-23T13:22:01.794 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 3452, 2024-03-23T13:22:01.794 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 26211650, 2024-03-23T13:22:01.794 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-03-23T13:22:01.794 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T13:22:01.794 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-03-23T13:22:01.794 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T13:22:01.794 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-03-23T13:22:01.795 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-03-23T13:22:01.795 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T13:22:01.795 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T13:22:01.795 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-03-23T13:22:01.795 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-03-23T13:22:01.795 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 21, 2024-03-23T13:22:01.795 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 797, 2024-03-23T13:22:01.795 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T13:22:01.795 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T13:22:01.796 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T13:22:01.796 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 907, 2024-03-23T13:22:01.796 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 775, 2024-03-23T13:22:01.796 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 13, 2024-03-23T13:22:01.796 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 922 2024-03-23T13:22:01.796 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.797 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-03-23T13:22:01.797 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-03-23T13:22:01.797 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-03-23T13:22:01.797 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-03-23T13:22:01.797 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 13, 2024-03-23T13:22:01.797 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 21, 2024-03-23T13:22:01.797 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-03-23T13:22:01.797 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-03-23T13:22:01.798 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-03-23T13:22:01.798 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 460, 2024-03-23T13:22:01.798 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 460, 2024-03-23T13:22:01.798 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-03-23T13:22:01.798 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-03-23T13:22:01.798 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-03-23T13:22:01.798 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-03-23T13:22:01.798 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-03-23T13:22:01.798 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-03-23T13:22:01.799 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-03-23T13:22:01.799 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 0, 2024-03-23T13:22:01.799 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 0, 2024-03-23T13:22:01.799 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T13:22:01.799 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-03-23T13:22:01.799 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-03-23T13:22:01.799 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-03-23T13:22:01.799 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-03-23T13:22:01.799 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-03-23T13:22:01.800 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.800 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-03-23T13:22:01.800 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-03-23T13:22:01.800 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 2670, 2024-03-23T13:22:01.800 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-03-23T13:22:01.800 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-03-23T13:22:01.800 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 2670, 2024-03-23T13:22:01.800 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-03-23T13:22:01.800 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-03-23T13:22:01.801 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 5, 2024-03-23T13:22:01.801 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-03-23T13:22:01.801 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-03-23T13:22:01.801 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 5, 2024-03-23T13:22:01.801 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 2, 2024-03-23T13:22:01.801 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-03-23T13:22:01.801 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-03-23T13:22:01.801 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-03-23T13:22:01.802 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 8447966, 2024-03-23T13:22:01.802 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-03-23T13:22:01.802 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-03-23T13:22:01.802 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 71, 2024-03-23T13:22:01.802 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.286767123, 2024-03-23T13:22:01.802 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004038973 2024-03-23T13:22:01.802 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.802 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-03-23T13:22:01.803 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.803 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-03-23T13:22:01.803 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 870, 2024-03-23T13:22:01.803 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 1330, 2024-03-23T13:22:01.803 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 460, 2024-03-23T13:22:01.803 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 66, 2024-03-23T13:22:01.803 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 67, 2024-03-23T13:22:01.803 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 1, 2024-03-23T13:22:01.803 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 1413, 2024-03-23T13:22:01.804 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 1886, 2024-03-23T13:22:01.804 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 473, 2024-03-23T13:22:01.804 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 268, 2024-03-23T13:22:01.804 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 1307, 2024-03-23T13:22:01.804 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 1039, 2024-03-23T13:22:01.804 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 63204, 2024-03-23T13:22:01.804 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 223612 2024-03-23T13:22:01.804 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.805 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-03-23T13:22:01.805 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 2310, 2024-03-23T13:22:01.805 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-03-23T13:22:01.805 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 1461, 2024-03-23T13:22:01.805 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 6, 2024-03-23T13:22:01.805 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 763, 2024-03-23T13:22:01.805 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-03-23T13:22:01.805 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 304, 2024-03-23T13:22:01.806 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.438651921, 2024-03-23T13:22:01.806 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001442933 2024-03-23T13:22:01.806 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.806 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-03-23T13:22:01.806 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.806 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.806 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.806 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.806 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-03-23T13:22:01.807 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.807 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.807 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.807 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.807 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-03-23T13:22:01.807 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.807 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.807 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.807 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.808 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-03-23T13:22:01.808 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 654, 2024-03-23T13:22:01.808 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.353501858, 2024-03-23T13:22:01.808 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000540522 2024-03-23T13:22:01.808 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.808 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-03-23T13:22:01.808 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.808 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.808 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.809 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.809 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-03-23T13:22:01.809 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-03-23T13:22:01.809 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000091119, 2024-03-23T13:22:01.809 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000091119 2024-03-23T13:22:01.809 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.809 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-03-23T13:22:01.809 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.809 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.810 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.810 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.810 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-03-23T13:22:01.810 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.810 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.810 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.810 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.810 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-03-23T13:22:01.811 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.811 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.811 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.811 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.811 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-03-23T13:22:01.811 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.811 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-03-23T13:22:01.812 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-03-23T13:22:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.813 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.814 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.814 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-03-23T13:22:01.814 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.814 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.814 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.814 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.814 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-03-23T13:22:01.814 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 18, 2024-03-23T13:22:01.815 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.004243871, 2024-03-23T13:22:01.815 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000235770 2024-03-23T13:22:01.815 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.815 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-03-23T13:22:01.815 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 11, 2024-03-23T13:22:01.815 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.009486013, 2024-03-23T13:22:01.815 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000862364 2024-03-23T13:22:01.815 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.816 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-03-23T13:22:01.816 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.816 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.816 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.816 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.816 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-03-23T13:22:01.816 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.816 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-03-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-03-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.818 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.818 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.818 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.818 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-03-23T13:22:01.818 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.818 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.818 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.818 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-03-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-03-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-03-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-03-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-03-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.822 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-03-23T13:22:01.822 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.822 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.822 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.822 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.822 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-03-23T13:22:01.822 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.822 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-03-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 460, 2024-03-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.326342355, 2024-03-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002883352 2024-03-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-03-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-03-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-03-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-03-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-03-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 2, 2024-03-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 1, 2024-03-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2024-03-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-03-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 1045, 2024-03-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 1045, 2024-03-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 137, 2024-03-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-03-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-03-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-03-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:22:01.828 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:22:01.828 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-03-23T13:22:01.828 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:22:01.828 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-03-23T13:22:01.828 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:22:01.828 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:22:01.828 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:22:01.828 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:22:01.829 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:22:01.829 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:22:01.829 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:22:01.829 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-03-23T13:22:01.829 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:22:01.829 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:22:01.829 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:22:01.829 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-03-23T13:22:01.830 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-03-23T13:22:01.830 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-03-23T13:22:01.830 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:22:01.830 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:22:01.830 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:22:01.830 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:22:01.830 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 330565, 2024-03-23T13:22:01.830 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 981, 2024-03-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 880, 2024-03-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 10, 2024-03-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-03-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-03-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-03-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-03-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 7528, 2024-03-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 158, 2024-03-23T13:22:01.832 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:22:01.832 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:22:01.832 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-03-23T13:22:01.832 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-03-23T13:22:01.832 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 3231366, 2024-03-23T13:22:01.832 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 37006, 2024-03-23T13:22:01.832 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-03-23T13:22:01.832 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-03-23T13:22:01.832 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-03-23T13:22:01.833 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-03-23T13:22:01.833 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.833 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-03-23T13:22:01.833 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-03-23T13:22:01.833 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-03-23T13:22:01.833 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 640, 2024-03-23T13:22:01.833 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 4399366, 2024-03-23T13:22:01.833 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 29, 2024-03-23T13:22:01.833 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 640, 2024-03-23T13:22:01.834 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-03-23T13:22:01.834 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 640, 2024-03-23T13:22:01.834 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 2.615944768, 2024-03-23T13:22:01.834 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004087413 2024-03-23T13:22:01.834 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.834 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-03-23T13:22:01.834 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-03-23T13:22:01.834 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 640, 2024-03-23T13:22:01.834 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 696 2024-03-23T13:22:01.835 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.835 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 640, 2024-03-23T13:22:01.835 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 13, 2024-03-23T13:22:01.835 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 627, 2024-03-23T13:22:01.835 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-03-23T13:22:01.835 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-03-23T13:22:01.835 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2024-03-23T13:22:01.835 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 10, 2024-03-23T13:22:01.836 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 13, 2024-03-23T13:22:01.836 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 84, 2024-03-23T13:22:01.836 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 19, 2024-03-23T13:22:01.836 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-03-23T13:22:01.836 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-03-23T13:22:01.836 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-03-23T13:22:01.836 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-03-23T13:22:01.836 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 473, 2024-03-23T13:22:01.837 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-03-23T13:22:01.837 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-03-23T13:22:01.837 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-03-23T13:22:01.837 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 0, 2024-03-23T13:22:01.837 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 20, 2024-03-23T13:22:01.837 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-03-23T13:22:01.837 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-03-23T13:22:01.837 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-03-23T13:22:01.838 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-03-23T13:22:01.838 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-03-23T13:22:01.838 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-03-23T13:22:01.838 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T13:22:01.838 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-03-23T13:22:01.838 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-03-23T13:22:01.838 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-03-23T13:22:01.838 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-03-23T13:22:01.838 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-03-23T13:22:01.839 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-03-23T13:22:01.839 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-03-23T13:22:01.839 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-03-23T13:22:01.839 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-03-23T13:22:01.839 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-03-23T13:22:01.839 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-03-23T13:22:01.839 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-03-23T13:22:01.839 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-03-23T13:22:01.840 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-03-23T13:22:01.840 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-03-23T13:22:01.840 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-03-23T13:22:01.840 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-03-23T13:22:01.840 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-03-23T13:22:01.840 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-03-23T13:22:01.840 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 87, 2024-03-23T13:22:01.840 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-03-23T13:22:01.841 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 2, 2024-03-23T13:22:01.841 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2024-03-23T13:22:01.841 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2024-03-23T13:22:01.841 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-03-23T13:22:01.841 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-03-23T13:22:01.841 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 67, 2024-03-23T13:22:01.841 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 0, 2024-03-23T13:22:01.841 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 10 2024-03-23T13:22:01.842 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.842 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-03-23T13:22:01.842 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-03-23T13:22:01.842 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-03-23T13:22:01.842 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-03-23T13:22:01.842 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-03-23T13:22:01.842 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.842 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-03-23T13:22:01.843 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-03-23T13:22:01.843 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.843 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-03-23T13:22:01.843 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-03-23T13:22:01.843 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 6, 2024-03-23T13:22:01.843 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-03-23T13:22:01.843 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 6, 2024-03-23T13:22:01.843 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 460, 2024-03-23T13:22:01.844 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 460, 2024-03-23T13:22:01.844 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-03-23T13:22:01.844 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.844 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:22:01.844 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:22:01.844 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-23T13:22:01.844 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:22:01.844 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 4839, 2024-03-23T13:22:01.844 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2749769, 2024-03-23T13:22:01.845 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:22:01.845 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 4839, 2024-03-23T13:22:01.845 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:22:01.845 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:22:01.845 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 4839, 2024-03-23T13:22:01.845 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2749769, 2024-03-23T13:22:01.845 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:22:01.845 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.846 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.846 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.846 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:01.846 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.846 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-03-23T13:22:01.846 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:22:01.846 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-23T13:22:01.846 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:22:01.847 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-23T13:22:01.847 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-23T13:22:01.847 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:22:01.847 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-23T13:22:01.847 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 640, 2024-03-23T13:22:01.847 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 4445826, 2024-03-23T13:22:01.847 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 640, 2024-03-23T13:22:01.847 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 4445826, 2024-03-23T13:22:01.848 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:22:01.848 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.848 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.848 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.848 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:01.848 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.848 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-03-23T13:22:01.848 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:22:01.849 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-03-23T13:22:01.849 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:22:01.849 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-23T13:22:01.849 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-23T13:22:01.849 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:22:01.849 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-23T13:22:01.849 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 640, 2024-03-23T13:22:01.849 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 640, 2024-03-23T13:22:01.850 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 640, 2024-03-23T13:22:01.850 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 640, 2024-03-23T13:22:01.850 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:22:01.850 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.850 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.850 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.850 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:01.850 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.851 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-03-23T13:22:01.851 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:22:01.851 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-23T13:22:01.851 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:22:01.851 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 460, 2024-03-23T13:22:01.851 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 46460, 2024-03-23T13:22:01.851 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:22:01.851 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 460, 2024-03-23T13:22:01.852 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:22:01.852 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:22:01.852 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 13, 2024-03-23T13:22:01.852 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 46460, 2024-03-23T13:22:01.852 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:22:01.852 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.852 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.852 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.853 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:01.853 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:01.853 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55f52cc30500": { 2024-03-23T13:22:01.853 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 5740, 2024-03-23T13:22:01.853 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-23T13:22:01.853 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:22:01.853 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 2670, 2024-03-23T13:22:01.853 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 4253662, 2024-03-23T13:22:01.854 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:22:01.854 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 2670, 2024-03-23T13:22:01.854 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:22:01.854 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:22:01.854 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 71, 2024-03-23T13:22:01.854 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 4247922, 2024-03-23T13:22:01.854 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:22:01.854 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:01.855 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:01.855 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:01.855 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:01.855 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:01.855 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-23T13:22:01.855 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:01 smithi007 ceph-mon[32184]: pgmap v416: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 64 KiB/s wr, 12 op/s 2024-03-23T13:22:02.765 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-23T13:22:02.765 INFO:tasks.check_counter:Daemon mds.i mds.imported=28 2024-03-23T13:22:02.765 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-03-23T13:22:02.765 DEBUG:tasks.check_counter:Getting stats from l 2024-03-23T13:22:02.766 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 745f591a-e916-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-23T13:22:03.030 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T13:22:03.427 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:03 smithi155 ceph-mon[36157]: pgmap v417: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-03-23T13:22:03.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:03 smithi130 ceph-mon[36225]: pgmap v417: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-03-23T13:22:03.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:03 smithi007 ceph-mon[32184]: pgmap v417: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-03-23T13:22:04.972 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-23T13:22:04.973 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T13:22:04.973 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 719, 2024-03-23T13:22:04.973 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 721, 2024-03-23T13:22:04.973 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 4459875, 2024-03-23T13:22:04.973 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 223616, 2024-03-23T13:22:04.974 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 6, 2024-03-23T13:22:04.974 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 2, 2024-03-23T13:22:04.974 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.047275261, 2024-03-23T13:22:04.974 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.013097059, 2024-03-23T13:22:04.974 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.014566638, 2024-03-23T13:22:04.974 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.015750483, 2024-03-23T13:22:04.974 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:22:04.974 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 719, 2024-03-23T13:22:04.975 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.016662430, 2024-03-23T13:22:04.975 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000023174 2024-03-23T13:22:04.975 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.975 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:22:04.975 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:04.975 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:04.975 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:04.976 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.976 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T13:22:04.976 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T13:22:04.976 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.976 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T13:22:04.976 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 7, 2024-03-23T13:22:04.976 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 67, 2024-03-23T13:22:04.976 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 3708, 2024-03-23T13:22:04.977 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 181468, 2024-03-23T13:22:04.977 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-03-23T13:22:04.977 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-03-23T13:22:04.977 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.003849252, 2024-03-23T13:22:04.977 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.001923253, 2024-03-23T13:22:04.977 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.001398619, 2024-03-23T13:22:04.977 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000115486, 2024-03-23T13:22:04.977 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:22:04.978 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 66, 2024-03-23T13:22:04.978 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.003401733, 2024-03-23T13:22:04.978 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000051541 2024-03-23T13:22:04.978 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.978 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:22:04.978 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:04.978 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:04.978 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:04.979 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.979 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-03-23T13:22:04.979 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 180784 2024-03-23T13:22:04.979 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.979 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T13:22:04.979 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 98, 2024-03-23T13:22:04.979 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 91, 2024-03-23T13:22:04.979 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 480380, 2024-03-23T13:22:04.979 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 51607, 2024-03-23T13:22:04.980 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 3, 2024-03-23T13:22:04.980 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 2, 2024-03-23T13:22:04.980 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.012685933, 2024-03-23T13:22:04.980 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.003472750, 2024-03-23T13:22:04.980 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.010305689, 2024-03-23T13:22:04.980 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.001529209, 2024-03-23T13:22:04.980 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T13:22:04.981 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 90, 2024-03-23T13:22:04.981 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.003700755, 2024-03-23T13:22:04.981 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000041119 2024-03-23T13:22:04.981 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.981 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-23T13:22:04.981 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:04.981 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:04.981 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:04.981 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.982 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 480112, 2024-03-23T13:22:04.982 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 51296 2024-03-23T13:22:04.982 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.982 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-03-23T13:22:04.982 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-03-23T13:22:04.982 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-03-23T13:22:04.982 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.983 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-03-23T13:22:04.983 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-23T13:22:04.983 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-23T13:22:04.983 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1261, 2024-03-23T13:22:04.983 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.026102117, 2024-03-23T13:22:04.983 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000020699 2024-03-23T13:22:04.983 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:04.983 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.984 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-03-23T13:22:04.984 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-23T13:22:04.984 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-23T13:22:04.984 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:04.984 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:04.984 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:04.984 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:04.984 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.985 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-03-23T13:22:04.985 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-03-23T13:22:04.985 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-03-23T13:22:04.985 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-03-23T13:22:04.985 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:04.985 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:04.985 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:04.985 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.986 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-03-23T13:22:04.986 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-03-23T13:22:04.986 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-03-23T13:22:04.986 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-03-23T13:22:04.986 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-03-23T13:22:04.986 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-03-23T13:22:04.986 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-03-23T13:22:04.986 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 1844, 2024-03-23T13:22:04.987 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 233, 2024-03-23T13:22:04.987 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 1611, 2024-03-23T13:22:04.987 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-03-23T13:22:04.987 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 1512, 2024-03-23T13:22:04.987 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-03-23T13:22:04.987 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-03-23T13:22:04.987 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-03-23T13:22:04.987 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 30, 2024-03-23T13:22:04.988 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-03-23T13:22:04.988 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-03-23T13:22:04.988 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-03-23T13:22:04.988 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-03-23T13:22:04.988 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-03-23T13:22:04.988 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-03-23T13:22:04.988 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-03-23T13:22:04.989 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-03-23T13:22:04.989 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-03-23T13:22:04.989 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-03-23T13:22:04.989 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-03-23T13:22:04.989 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-03-23T13:22:04.989 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-03-23T13:22:04.989 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-03-23T13:22:04.989 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-03-23T13:22:04.989 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-03-23T13:22:04.990 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 0, 2024-03-23T13:22:04.990 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 0, 2024-03-23T13:22:04.990 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-03-23T13:22:04.990 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T13:22:04.990 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-03-23T13:22:04.990 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T13:22:04.990 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-03-23T13:22:04.991 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-03-23T13:22:04.991 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T13:22:04.991 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T13:22:04.991 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-03-23T13:22:04.991 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-03-23T13:22:04.991 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T13:22:04.991 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-03-23T13:22:04.991 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T13:22:04.992 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T13:22:04.992 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T13:22:04.992 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-03-23T13:22:04.992 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-03-23T13:22:04.992 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-03-23T13:22:04.992 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-03-23T13:22:04.992 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.992 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-03-23T13:22:04.993 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-03-23T13:22:04.993 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-03-23T13:22:04.993 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-03-23T13:22:04.993 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-03-23T13:22:04.993 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-03-23T13:22:04.993 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-03-23T13:22:04.993 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-03-23T13:22:04.993 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-03-23T13:22:04.993 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-03-23T13:22:04.994 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-03-23T13:22:04.994 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-03-23T13:22:04.994 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-03-23T13:22:04.994 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-03-23T13:22:04.994 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-03-23T13:22:04.994 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-03-23T13:22:04.994 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-03-23T13:22:04.994 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-03-23T13:22:04.994 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 0, 2024-03-23T13:22:04.995 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 0, 2024-03-23T13:22:04.995 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T13:22:04.995 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-03-23T13:22:04.995 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-03-23T13:22:04.995 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-03-23T13:22:04.995 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-03-23T13:22:04.995 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-03-23T13:22:04.995 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.996 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-03-23T13:22:04.996 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-03-23T13:22:04.996 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-03-23T13:22:04.996 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-03-23T13:22:04.996 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-03-23T13:22:04.996 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 2670, 2024-03-23T13:22:04.996 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-03-23T13:22:04.996 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-03-23T13:22:04.996 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-03-23T13:22:04.997 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-03-23T13:22:04.997 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-03-23T13:22:04.997 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 5, 2024-03-23T13:22:04.997 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 2, 2024-03-23T13:22:04.997 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-03-23T13:22:04.997 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-03-23T13:22:04.997 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-03-23T13:22:04.997 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 8447966, 2024-03-23T13:22:04.997 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 8447392, 2024-03-23T13:22:04.998 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-03-23T13:22:04.998 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:04.998 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:04.998 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:04.998 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.998 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 2670 2024-03-23T13:22:04.998 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:04.998 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-03-23T13:22:04.999 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 862, 2024-03-23T13:22:04.999 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 1322, 2024-03-23T13:22:04.999 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 460, 2024-03-23T13:22:04.999 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 58, 2024-03-23T13:22:04.999 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 58, 2024-03-23T13:22:04.999 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 0, 2024-03-23T13:22:04.999 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 1844, 2024-03-23T13:22:04.999 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 1844, 2024-03-23T13:22:05.000 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 0, 2024-03-23T13:22:05.000 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-03-23T13:22:05.000 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-03-23T13:22:05.000 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-03-23T13:22:05.000 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 52196, 2024-03-23T13:22:05.000 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 215420 2024-03-23T13:22:05.000 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.000 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-03-23T13:22:05.000 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-03-23T13:22:05.001 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-03-23T13:22:05.001 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-03-23T13:22:05.001 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-03-23T13:22:05.001 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-03-23T13:22:05.001 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-03-23T13:22:05.001 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.001 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.001 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.002 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.002 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-03-23T13:22:05.002 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.002 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.002 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.002 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.002 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-03-23T13:22:05.002 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.003 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.003 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.003 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.003 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-03-23T13:22:05.003 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.003 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.003 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.003 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.004 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-03-23T13:22:05.004 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.004 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.004 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.004 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.004 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-03-23T13:22:05.004 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.004 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.004 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.005 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.005 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-03-23T13:22:05.005 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.005 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.005 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.005 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.005 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-03-23T13:22:05.005 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.006 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.006 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.006 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.006 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-03-23T13:22:05.006 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.006 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.006 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.006 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.007 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-03-23T13:22:05.007 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.007 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.007 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.007 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.007 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-03-23T13:22:05.007 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.007 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.007 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.008 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.008 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-03-23T13:22:05.008 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.009 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.009 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.009 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.009 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-03-23T13:22:05.009 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.009 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.009 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.010 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.010 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-03-23T13:22:05.010 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.010 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.010 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.010 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.010 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-03-23T13:22:05.010 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.010 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.011 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.011 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.011 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-03-23T13:22:05.011 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.011 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.011 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.011 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.011 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-03-23T13:22:05.012 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.012 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.012 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.012 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.012 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-03-23T13:22:05.012 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.012 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.012 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.012 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.013 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-03-23T13:22:05.013 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.013 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.013 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.013 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.013 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-03-23T13:22:05.013 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.013 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.014 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.014 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.014 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-03-23T13:22:05.014 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.014 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.014 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.014 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.014 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-03-23T13:22:05.014 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.015 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.015 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.015 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.015 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-03-23T13:22:05.015 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.015 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.015 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.015 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.016 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-03-23T13:22:05.016 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.016 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.016 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.016 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.016 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-03-23T13:22:05.016 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.016 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.016 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.017 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.017 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-03-23T13:22:05.017 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.017 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.017 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.017 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.017 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-03-23T13:22:05.017 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.018 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.018 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.018 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.018 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-03-23T13:22:05.018 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.018 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.018 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.018 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.019 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-03-23T13:22:05.019 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.019 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.019 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.019 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.019 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-03-23T13:22:05.019 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-03-23T13:22:05.019 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-03-23T13:22:05.020 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.020 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.020 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.020 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:05.020 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.020 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-03-23T13:22:05.020 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-03-23T13:22:05.020 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 2, 2024-03-23T13:22:05.021 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 1, 2024-03-23T13:22:05.021 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2024-03-23T13:22:05.021 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-03-23T13:22:05.021 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-03-23T13:22:05.021 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-03-23T13:22:05.021 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1492, 2024-03-23T13:22:05.021 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-03-23T13:22:05.021 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.022 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-03-23T13:22:05.022 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-03-23T13:22:05.022 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-03-23T13:22:05.022 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T13:22:05.022 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-03-23T13:22:05.022 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T13:22:05.022 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-03-23T13:22:05.023 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T13:22:05.023 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T13:22:05.023 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T13:22:05.023 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T13:22:05.023 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T13:22:05.023 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-03-23T13:22:05.023 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T13:22:05.023 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T13:22:05.024 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-03-23T13:22:05.024 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-03-23T13:22:05.024 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-03-23T13:22:05.024 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-03-23T13:22:05.024 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T13:22:05.024 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T13:22:05.024 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T13:22:05.024 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T13:22:05.025 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T13:22:05.025 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-03-23T13:22:05.025 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-03-23T13:22:05.025 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-03-23T13:22:05.025 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T13:22:05.025 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T13:22:05.025 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-03-23T13:22:05.025 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-03-23T13:22:05.025 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-03-23T13:22:05.026 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-03-23T13:22:05.026 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T13:22:05.026 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-03-23T13:22:05.026 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T13:22:05.026 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-03-23T13:22:05.026 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 212737, 2024-03-23T13:22:05.026 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 898, 2024-03-23T13:22:05.027 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-03-23T13:22:05.027 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-03-23T13:22:05.027 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-03-23T13:22:05.027 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-03-23T13:22:05.027 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-03-23T13:22:05.027 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-03-23T13:22:05.027 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-03-23T13:22:05.027 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-03-23T13:22:05.028 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 7528, 2024-03-23T13:22:05.028 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 158, 2024-03-23T13:22:05.028 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T13:22:05.028 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-03-23T13:22:05.029 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-03-23T13:22:05.029 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-03-23T13:22:05.029 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 3324247, 2024-03-23T13:22:05.029 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 28133, 2024-03-23T13:22:05.029 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-03-23T13:22:05.029 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-03-23T13:22:05.030 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-03-23T13:22:05.030 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-03-23T13:22:05.030 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.030 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-03-23T13:22:05.030 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-03-23T13:22:05.030 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-03-23T13:22:05.030 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 722, 2024-03-23T13:22:05.030 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 21, 2024-03-23T13:22:05.031 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 3, 2024-03-23T13:22:05.031 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 722, 2024-03-23T13:22:05.031 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-03-23T13:22:05.031 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 722, 2024-03-23T13:22:05.031 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.550988360, 2024-03-23T13:22:05.031 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000763141 2024-03-23T13:22:05.031 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.032 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-03-23T13:22:05.032 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-03-23T13:22:05.032 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 722, 2024-03-23T13:22:05.032 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 723 2024-03-23T13:22:05.032 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.032 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 722, 2024-03-23T13:22:05.032 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 722, 2024-03-23T13:22:05.033 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-03-23T13:22:05.033 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-03-23T13:22:05.033 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-03-23T13:22:05.033 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 428, 2024-03-23T13:22:05.033 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-03-23T13:22:05.033 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 292, 2024-03-23T13:22:05.034 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-03-23T13:22:05.034 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-03-23T13:22:05.034 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-03-23T13:22:05.034 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-03-23T13:22:05.034 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-03-23T13:22:05.034 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-03-23T13:22:05.034 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-03-23T13:22:05.035 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-03-23T13:22:05.035 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-03-23T13:22:05.035 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-03-23T13:22:05.035 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 1, 2024-03-23T13:22:05.035 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-03-23T13:22:05.035 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-03-23T13:22:05.035 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-03-23T13:22:05.035 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-03-23T13:22:05.036 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-03-23T13:22:05.036 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-03-23T13:22:05.036 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-03-23T13:22:05.036 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T13:22:05.036 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-03-23T13:22:05.036 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-03-23T13:22:05.036 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-03-23T13:22:05.036 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-03-23T13:22:05.036 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-03-23T13:22:05.037 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-03-23T13:22:05.037 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-03-23T13:22:05.037 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-03-23T13:22:05.037 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-03-23T13:22:05.037 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-03-23T13:22:05.037 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-03-23T13:22:05.037 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-03-23T13:22:05.037 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-03-23T13:22:05.037 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-03-23T13:22:05.038 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-03-23T13:22:05.038 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-03-23T13:22:05.038 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-03-23T13:22:05.038 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-03-23T13:22:05.038 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-03-23T13:22:05.038 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 87, 2024-03-23T13:22:05.038 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-03-23T13:22:05.038 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 2, 2024-03-23T13:22:05.039 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 4, 2024-03-23T13:22:05.039 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 4, 2024-03-23T13:22:05.039 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-03-23T13:22:05.039 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-03-23T13:22:05.039 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-03-23T13:22:05.039 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-03-23T13:22:05.039 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-03-23T13:22:05.039 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.040 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-03-23T13:22:05.040 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-03-23T13:22:05.040 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-03-23T13:22:05.040 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-03-23T13:22:05.040 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-03-23T13:22:05.040 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.040 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-03-23T13:22:05.040 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-03-23T13:22:05.040 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.041 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-03-23T13:22:05.041 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-03-23T13:22:05.041 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T13:22:05.041 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-03-23T13:22:05.041 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-03-23T13:22:05.041 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-03-23T13:22:05.041 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-03-23T13:22:05.041 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-03-23T13:22:05.042 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.042 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T13:22:05.042 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:22:05.042 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-23T13:22:05.042 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:22:05.042 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 820, 2024-03-23T13:22:05.042 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 4862205, 2024-03-23T13:22:05.043 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:22:05.043 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 820, 2024-03-23T13:22:05.043 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:22:05.043 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:22:05.043 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 820, 2024-03-23T13:22:05.043 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 4862205, 2024-03-23T13:22:05.043 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:22:05.043 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.044 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.044 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.044 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:05.044 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.044 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-03-23T13:22:05.044 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:22:05.044 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-23T13:22:05.045 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:22:05.045 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-23T13:22:05.045 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-23T13:22:05.045 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:22:05.045 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-23T13:22:05.045 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 722, 2024-03-23T13:22:05.046 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 4255071, 2024-03-23T13:22:05.046 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 722, 2024-03-23T13:22:05.046 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 4255071, 2024-03-23T13:22:05.046 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:22:05.046 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.046 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.047 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.047 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:05.047 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.047 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-03-23T13:22:05.047 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:22:05.047 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-03-23T13:22:05.047 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:22:05.047 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-23T13:22:05.048 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-23T13:22:05.048 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:22:05.048 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-23T13:22:05.048 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 722, 2024-03-23T13:22:05.048 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 722, 2024-03-23T13:22:05.048 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 722, 2024-03-23T13:22:05.048 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 722, 2024-03-23T13:22:05.048 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:22:05.049 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.049 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.049 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.049 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:05.049 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.049 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-03-23T13:22:05.049 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:22:05.050 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-23T13:22:05.050 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:22:05.050 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-23T13:22:05.050 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-23T13:22:05.050 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:22:05.050 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-23T13:22:05.050 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:22:05.051 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:22:05.051 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-03-23T13:22:05.051 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-03-23T13:22:05.051 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:22:05.051 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.051 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.051 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.052 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:05.052 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-23T13:22:05.052 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x564cb28cc6e0": { 2024-03-23T13:22:05.052 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-23T13:22:05.052 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-23T13:22:05.052 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-23T13:22:05.052 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-23T13:22:05.052 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-23T13:22:05.053 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-23T13:22:05.053 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-23T13:22:05.053 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-23T13:22:05.053 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-23T13:22:05.053 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-03-23T13:22:05.053 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-03-23T13:22:05.053 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-23T13:22:05.053 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-23T13:22:05.054 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-23T13:22:05.054 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-23T13:22:05.054 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:05.054 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-23T13:22:05.054 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-23T13:22:05.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:05 smithi130 ceph-mon[36225]: pgmap v418: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 17 op/s 2024-03-23T13:22:05.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:05 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:22:05.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:05 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:05 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:05 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:05 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.744 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:05 smithi155 ceph-mon[36157]: pgmap v418: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 17 op/s 2024-03-23T13:22:05.744 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:05 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:22:05.745 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:05 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.745 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:05 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.745 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:05 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.745 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:05 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.779 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-23T13:22:05.779 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-23T13:22:05.779 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-03-23T13:22:05.780 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-23T13:22:05.790 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-23T13:22:05.790 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:22:05.791 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T13:22:05.807 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-23T13:22:05.807 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T13:22:05.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:05 smithi007 ceph-mon[32184]: pgmap v418: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 17 op/s 2024-03-23T13:22:05.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T13:22:05.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:05 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:05.846 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:22:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.007+0000 7f3237e0e640 1 -- 172.21.15.7:0/3190442320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32301006b0 msgr2=0x7f3230100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.007+0000 7f3237e0e640 1 --2- 172.21.15.7:0/3190442320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32301006b0 0x7f3230100ad0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3228009ca0 tx=0x7f322802c0f0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.007+0000 7f3237e0e640 1 -- 172.21.15.7:0/3190442320 shutdown_connections 2024-03-23T13:22:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.007+0000 7f3237e0e640 1 --2- 172.21.15.7:0/3190442320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3230102260 0x7f3230108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.007+0000 7f3237e0e640 1 --2- 172.21.15.7:0/3190442320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32301018a0 0x7f3230101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.007+0000 7f3237e0e640 1 --2- 172.21.15.7:0/3190442320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32301006b0 0x7f3230100ad0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.007+0000 7f3237e0e640 1 -- 172.21.15.7:0/3190442320 >> 172.21.15.7:0/3190442320 conn(0x7f32300fbc40 msgr2=0x7f32300fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.007+0000 7f3237e0e640 1 -- 172.21.15.7:0/3190442320 shutdown_connections 2024-03-23T13:22:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.007+0000 7f3237e0e640 1 -- 172.21.15.7:0/3190442320 wait complete. 2024-03-23T13:22:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f3237e0e640 1 Processor -- start 2024-03-23T13:22:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f3237e0e640 1 -- start start 2024-03-23T13:22:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f3237e0e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32301006b0 0x7f3230104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f3237e0e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32301018a0 0x7f3230102c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f3237e0e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3230102260 0x7f32301031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f3237e0e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3230103920 con 0x7f3230102260 2024-03-23T13:22:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f3237e0e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3230103a90 con 0x7f32301006b0 2024-03-23T13:22:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f3237e0e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32301a18e0 con 0x7f32301018a0 2024-03-23T13:22:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f32363ab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3230102260 0x7f32301031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f3235baa640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32301006b0 0x7f3230104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f32363ab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3230102260 0x7f32301031c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34040/0 (socket says 172.21.15.7:34040) 2024-03-23T13:22:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f32363ab640 1 -- 172.21.15.7:0/3419258001 learned_addr learned my addr 172.21.15.7:0/3419258001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f32353a9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32301018a0 0x7f3230102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f32363ab640 1 -- 172.21.15.7:0/3419258001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32301018a0 msgr2=0x7f3230102c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f32363ab640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32301018a0 0x7f3230102c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f32363ab640 1 -- 172.21.15.7:0/3419258001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32301006b0 msgr2=0x7f3230104640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f32363ab640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32301006b0 0x7f3230104640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.008+0000 7f32363ab640 1 -- 172.21.15.7:0/3419258001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3228009900 con 0x7f3230102260 2024-03-23T13:22:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.009+0000 7f32363ab640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3230102260 0x7f32301031c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9a0 tx=0x7f322000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.009+0000 7f322effd640 1 -- 172.21.15.7:0/3419258001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322000ccc0 con 0x7f3230102260 2024-03-23T13:22:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.009+0000 7f322effd640 1 -- 172.21.15.7:0/3419258001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f322000ce20 con 0x7f3230102260 2024-03-23T13:22:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.009+0000 7f3237e0e640 1 -- 172.21.15.7:0/3419258001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32301a1b90 con 0x7f3230102260 2024-03-23T13:22:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.009+0000 7f322effd640 1 -- 172.21.15.7:0/3419258001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220010690 con 0x7f3230102260 2024-03-23T13:22:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.009+0000 7f3237e0e640 1 -- 172.21.15.7:0/3419258001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301a20a0 con 0x7f3230102260 2024-03-23T13:22:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.010+0000 7f3237e0e640 1 -- 172.21.15.7:0/3419258001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f323010c050 con 0x7f3230102260 2024-03-23T13:22:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.011+0000 7f322effd640 1 -- 172.21.15.7:0/3419258001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f32200046c0 con 0x7f3230102260 2024-03-23T13:22:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.011+0000 7f322effd640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32180a20c0 0x7f32180a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.011+0000 7f3235baa640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32180a20c0 0x7f32180a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.011+0000 7f322effd640 1 -- 172.21.15.7:0/3419258001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3220014070 con 0x7f3230102260 2024-03-23T13:22:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.012+0000 7f3235baa640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32180a20c0 0x7f32180a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3228005e30 tx=0x7f32280326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.016+0000 7f322effd640 1 -- 172.21.15.7:0/3419258001 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f3220096f60 con 0x7f3230102260 2024-03-23T13:22:06.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.202+0000 7f3237e0e640 1 -- 172.21.15.7:0/3419258001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f323010c1e0 con 0x7f3230102260 2024-03-23T13:22:06.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.204+0000 7f322effd640 1 -- 172.21.15.7:0/3419258001 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22558 (secure 0 0 0) 0x7f322009a620 con 0x7f3230102260 2024-03-23T13:22:06.208 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:22:06.208 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:18:43.667397+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:22:06.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 -- 172.21.15.7:0/3419258001 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32180a20c0 msgr2=0x7f32180a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32180a20c0 0x7f32180a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3228005e30 tx=0x7f32280326b0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 -- 172.21.15.7:0/3419258001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3230102260 msgr2=0x7f32301031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3230102260 0x7f32301031c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9a0 tx=0x7f322000eda0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 -- 172.21.15.7:0/3419258001 shutdown_connections 2024-03-23T13:22:06.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32180a20c0 0x7f32180a4580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3230102260 0x7f32301031c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32301018a0 0x7f3230102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 --2- 172.21.15.7:0/3419258001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32301006b0 0x7f3230104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 -- 172.21.15.7:0/3419258001 >> 172.21.15.7:0/3419258001 conn(0x7f32300fbc40 msgr2=0x7f32300fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:06.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 -- 172.21.15.7:0/3419258001 shutdown_connections 2024-03-23T13:22:06.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.206+0000 7f3237e0e640 1 -- 172.21.15.7:0/3419258001 wait complete. 2024-03-23T13:22:06.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T13:13:08.186719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T13:17:38.843887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T13:17:39.788489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T13:22:06.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T13:22:06.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.323+0000 7f3242381640 1 -- 172.21.15.7:0/2236142712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f323c101810 msgr2=0x7f323c101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.323+0000 7f3242381640 1 --2- 172.21.15.7:0/2236142712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f323c101810 0x7f323c101c90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3230009900 tx=0x7f323002bc30 comp rx=0 tx=0).stop 2024-03-23T13:22:06.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.323+0000 7f3242381640 1 -- 172.21.15.7:0/2236142712 shutdown_connections 2024-03-23T13:22:06.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.323+0000 7f3242381640 1 --2- 172.21.15.7:0/2236142712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f323c1021d0 0x7f323c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.323+0000 7f3242381640 1 --2- 172.21.15.7:0/2236142712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f323c101810 0x7f323c101c90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.323+0000 7f3242381640 1 --2- 172.21.15.7:0/2236142712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f323c100620 0x7f323c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.323+0000 7f3242381640 1 -- 172.21.15.7:0/2236142712 >> 172.21.15.7:0/2236142712 conn(0x7f323c0fbbb0 msgr2=0x7f323c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:06.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.323+0000 7f3242381640 1 -- 172.21.15.7:0/2236142712 shutdown_connections 2024-03-23T13:22:06.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.323+0000 7f3242381640 1 -- 172.21.15.7:0/2236142712 wait complete. 2024-03-23T13:22:06.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.324+0000 7f3242381640 1 Processor -- start 2024-03-23T13:22:06.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.324+0000 7f3242381640 1 -- start start 2024-03-23T13:22:06.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.324+0000 7f3242381640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f323c100620 0x7f323c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.324+0000 7f3242381640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f323c101810 0x7f323c197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.324+0000 7f3242381640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f323c1021d0 0x7f323c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.324+0000 7f3242381640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f323c19f290 con 0x7f323c101810 2024-03-23T13:22:06.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.324+0000 7f3242381640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f323c19f400 con 0x7f323c100620 2024-03-23T13:22:06.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.324+0000 7f3242381640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f323c19f5c0 con 0x7f323c1021d0 2024-03-23T13:22:06.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f323b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f323c101810 0x7f323c197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f323b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f323c101810 0x7f323c197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34056/0 (socket says 172.21.15.7:34056) 2024-03-23T13:22:06.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f323b7fe640 1 -- 172.21.15.7:0/1806470678 learned_addr learned my addr 172.21.15.7:0/1806470678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:06.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f323bfff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f323c100620 0x7f323c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f324091e640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f323c1021d0 0x7f323c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f323b7fe640 1 -- 172.21.15.7:0/1806470678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f323c1021d0 msgr2=0x7f323c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f323b7fe640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f323c1021d0 0x7f323c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f323b7fe640 1 -- 172.21.15.7:0/1806470678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f323c100620 msgr2=0x7f323c1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f323b7fe640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f323c100620 0x7f323c1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f323b7fe640 1 -- 172.21.15.7:0/1806470678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3230009560 con 0x7f323c101810 2024-03-23T13:22:06.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f323b7fe640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f323c101810 0x7f323c197ce0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3230006010 tx=0x7f32300040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:06.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f32397fa640 1 -- 172.21.15.7:0/1806470678 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32300044c0 con 0x7f323c101810 2024-03-23T13:22:06.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f3242381640 1 -- 172.21.15.7:0/1806470678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323c19f840 con 0x7f323c101810 2024-03-23T13:22:06.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f32397fa640 1 -- 172.21.15.7:0/1806470678 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32300052d0 con 0x7f323c101810 2024-03-23T13:22:06.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f32397fa640 1 -- 172.21.15.7:0/1806470678 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230003e10 con 0x7f323c101810 2024-03-23T13:22:06.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.325+0000 7f3242381640 1 -- 172.21.15.7:0/1806470678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323c19fd50 con 0x7f323c101810 2024-03-23T13:22:06.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.326+0000 7f3242381640 1 -- 172.21.15.7:0/1806470678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f323c10c030 con 0x7f323c101810 2024-03-23T13:22:06.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.327+0000 7f32397fa640 1 -- 172.21.15.7:0/1806470678 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f3230005440 con 0x7f323c101810 2024-03-23T13:22:06.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.328+0000 7f32397fa640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32240a20c0 0x7f32240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.328+0000 7f323bfff640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32240a20c0 0x7f32240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.328+0000 7f32397fa640 1 -- 172.21.15.7:0/1806470678 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f32300effa0 con 0x7f323c101810 2024-03-23T13:22:06.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.328+0000 7f323bfff640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32240a20c0 0x7f32240a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f323c101670 tx=0x7f322c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:06.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.332+0000 7f32397fa640 1 -- 172.21.15.7:0/1806470678 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f32300b8f80 con 0x7f323c101810 2024-03-23T13:22:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.519+0000 7f3242381640 1 -- 172.21.15.7:0/1806470678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f323c1a01c0 con 0x7f323c101810 2024-03-23T13:22:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.519+0000 7f32397fa640 1 -- 172.21.15.7:0/1806470678 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f32300bc640 con 0x7f323c101810 2024-03-23T13:22:06.523 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-23T13:22:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.521+0000 7f3242381640 1 -- 172.21.15.7:0/1806470678 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32240a20c0 msgr2=0x7f32240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.521+0000 7f3242381640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32240a20c0 0x7f32240a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f323c101670 tx=0x7f322c005f90 comp rx=0 tx=0).stop 2024-03-23T13:22:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.521+0000 7f3242381640 1 -- 172.21.15.7:0/1806470678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f323c101810 msgr2=0x7f323c197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.521+0000 7f3242381640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f323c101810 0x7f323c197ce0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3230006010 tx=0x7f32300040c0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.521+0000 7f3242381640 1 -- 172.21.15.7:0/1806470678 shutdown_connections 2024-03-23T13:22:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.521+0000 7f3242381640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f32240a20c0 0x7f32240a4580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.521+0000 7f3242381640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f323c1021d0 0x7f323c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.521+0000 7f3242381640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f323c101810 0x7f323c197ce0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.521+0000 7f3242381640 1 --2- 172.21.15.7:0/1806470678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f323c100620 0x7f323c1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.521+0000 7f3242381640 1 -- 172.21.15.7:0/1806470678 >> 172.21.15.7:0/1806470678 conn(0x7f323c0fbbb0 msgr2=0x7f323c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.522+0000 7f3242381640 1 -- 172.21.15.7:0/1806470678 shutdown_connections 2024-03-23T13:22:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.522+0000 7f3242381640 1 -- 172.21.15.7:0/1806470678 wait complete. 2024-03-23T13:22:06.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:22:06.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:22:06.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:06 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:06.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:06 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3419258001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:22:06.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T13:22:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.638+0000 7f01d8842640 1 -- 172.21.15.7:0/3702510517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01d41021c0 msgr2=0x7f01d4108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.638+0000 7f01d8842640 1 --2- 172.21.15.7:0/3702510517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01d41021c0 0x7f01d4108a50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009ca0 tx=0x7f01cc02c0f0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.638+0000 7f01d8842640 1 -- 172.21.15.7:0/3702510517 shutdown_connections 2024-03-23T13:22:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.638+0000 7f01d8842640 1 --2- 172.21.15.7:0/3702510517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01d41021c0 0x7f01d4108a50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.638+0000 7f01d8842640 1 --2- 172.21.15.7:0/3702510517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d4101800 0x7f01d4101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.638+0000 7f01d8842640 1 --2- 172.21.15.7:0/3702510517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01d4100610 0x7f01d4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.638+0000 7f01d8842640 1 -- 172.21.15.7:0/3702510517 >> 172.21.15.7:0/3702510517 conn(0x7f01d40fbba0 msgr2=0x7f01d40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d8842640 1 -- 172.21.15.7:0/3702510517 shutdown_connections 2024-03-23T13:22:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d8842640 1 -- 172.21.15.7:0/3702510517 wait complete. 2024-03-23T13:22:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d8842640 1 Processor -- start 2024-03-23T13:22:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d8842640 1 -- start start 2024-03-23T13:22:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d8842640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01d4100610 0x7f01d4197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d8842640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01d4101800 0x7f01d4197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d8842640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d41021c0 0x7f01d419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d8842640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f01d419f260 con 0x7f01d4100610 2024-03-23T13:22:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d8842640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f01d419f3d0 con 0x7f01d41021c0 2024-03-23T13:22:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d8842640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01d419f590 con 0x7f01d4101800 2024-03-23T13:22:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01d4100610 0x7f01d4197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01d4100610 0x7f01d4197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34074/0 (socket says 172.21.15.7:34074) 2024-03-23T13:22:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.639+0000 7f01d259c640 1 -- 172.21.15.7:0/1101743027 learned_addr learned my addr 172.21.15.7:0/1101743027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01cbfff640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01d4101800 0x7f01d4197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01d2d9d640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d41021c0 0x7f01d419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01d259c640 1 -- 172.21.15.7:0/1101743027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01d4101800 msgr2=0x7f01d4197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01d259c640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01d4101800 0x7f01d4197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01d259c640 1 -- 172.21.15.7:0/1101743027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d41021c0 msgr2=0x7f01d419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01d259c640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d41021c0 0x7f01d419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01d259c640 1 -- 172.21.15.7:0/1101743027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01cc009900 con 0x7f01d4100610 2024-03-23T13:22:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01d2d9d640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d41021c0 0x7f01d419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:22:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01d259c640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01d4100610 0x7f01d4197770 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f01c400e9c0 tx=0x7f01c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01cb7fe640 1 -- 172.21.15.7:0/1101743027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c40096e0 con 0x7f01d4100610 2024-03-23T13:22:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01d8842640 1 -- 172.21.15.7:0/1101743027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d419f870 con 0x7f01d4100610 2024-03-23T13:22:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01cb7fe640 1 -- 172.21.15.7:0/1101743027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c400cd80 con 0x7f01d4100610 2024-03-23T13:22:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01cb7fe640 1 -- 172.21.15.7:0/1101743027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4010430 con 0x7f01d4100610 2024-03-23T13:22:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.640+0000 7f01d8842640 1 -- 172.21.15.7:0/1101743027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d419fd30 con 0x7f01d4100610 2024-03-23T13:22:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.641+0000 7f01d8842640 1 -- 172.21.15.7:0/1101743027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d4107930 con 0x7f01d4100610 2024-03-23T13:22:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.642+0000 7f01cb7fe640 1 -- 172.21.15.7:0/1101743027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f01c4003680 con 0x7f01d4100610 2024-03-23T13:22:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.643+0000 7f01cb7fe640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01c00a20c0 0x7f01c00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.643+0000 7f01cbfff640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01c00a20c0 0x7f01c00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.643+0000 7f01cb7fe640 1 -- 172.21.15.7:0/1101743027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f01c4014070 con 0x7f01d4100610 2024-03-23T13:22:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.643+0000 7f01cbfff640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01c00a20c0 0x7f01c00a4580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f01d4198c80 tx=0x7f01bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.646+0000 7f01cb7fe640 1 -- 172.21.15.7:0/1101743027 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f01c4097180 con 0x7f01d4100610 2024-03-23T13:22:06.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:22:06.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:22:06.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:06 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:06.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:06 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3419258001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:22:06.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T13:22:06.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T13:22:06.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:06 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' 2024-03-23T13:22:06.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:06 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3419258001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:22:06.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.827+0000 7f01d8842640 1 -- 172.21.15.7:0/1101743027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f01d410d120 con 0x7f01d4100610 2024-03-23T13:22:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.827+0000 7f01cb7fe640 1 -- 172.21.15.7:0/1101743027 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f01c409a840 con 0x7f01d4100610 2024-03-23T13:22:06.831 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-23T13:22:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 -- 172.21.15.7:0/1101743027 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01c00a20c0 msgr2=0x7f01c00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01c00a20c0 0x7f01c00a4580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f01d4198c80 tx=0x7f01bc00b410 comp rx=0 tx=0).stop 2024-03-23T13:22:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 -- 172.21.15.7:0/1101743027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01d4100610 msgr2=0x7f01d4197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01d4100610 0x7f01d4197770 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f01c400e9c0 tx=0x7f01c400ee80 comp rx=0 tx=0).stop 2024-03-23T13:22:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 -- 172.21.15.7:0/1101743027 shutdown_connections 2024-03-23T13:22:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f01c00a20c0 0x7f01c00a4580 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f01d41021c0 0x7f01d419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01d4101800 0x7f01d4197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 --2- 172.21.15.7:0/1101743027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01d4100610 0x7f01d4197770 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 -- 172.21.15.7:0/1101743027 >> 172.21.15.7:0/1101743027 conn(0x7f01d40fbba0 msgr2=0x7f01d4104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 -- 172.21.15.7:0/1101743027 shutdown_connections 2024-03-23T13:22:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.829+0000 7f01d8842640 1 -- 172.21.15.7:0/1101743027 wait complete. 2024-03-23T13:22:06.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.946+0000 7fbe417d6640 1 -- 172.21.15.7:0/1180642450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c101710 msgr2=0x7fbe3c101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.946+0000 7fbe417d6640 1 --2- 172.21.15.7:0/1180642450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c101710 0x7fbe3c101b30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009860 tx=0x7fbe3402bb70 comp rx=0 tx=0).stop 2024-03-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 -- 172.21.15.7:0/1180642450 shutdown_connections 2024-03-23T13:22:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 --2- 172.21.15.7:0/1180642450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c1032c0 0x7fbe3c109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 --2- 172.21.15.7:0/1180642450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c102900 0x7fbe3c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 --2- 172.21.15.7:0/1180642450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c101710 0x7fbe3c101b30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 -- 172.21.15.7:0/1180642450 >> 172.21.15.7:0/1180642450 conn(0x7fbe3c0fcc80 msgr2=0x7fbe3c0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 -- 172.21.15.7:0/1180642450 shutdown_connections 2024-03-23T13:22:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 -- 172.21.15.7:0/1180642450 wait complete. 2024-03-23T13:22:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 Processor -- start 2024-03-23T13:22:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 -- start start 2024-03-23T13:22:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101710 0x7fbe3c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c102900 0x7fbe3c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1032c0 0x7fbe3c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c19f2d0 con 0x7fbe3c102900 2024-03-23T13:22:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c19f440 con 0x7fbe3c101710 2024-03-23T13:22:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.947+0000 7fbe417d6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c19f600 con 0x7fbe3c1032c0 2024-03-23T13:22:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c102900 0x7fbe3c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c102900 0x7fbe3c197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34084/0 (socket says 172.21.15.7:34084) 2024-03-23T13:22:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3a7fc640 1 -- 172.21.15.7:0/3307438683 learned_addr learned my addr 172.21.15.7:0/3307438683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3b7fe640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1032c0 0x7fbe3c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3affd640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101710 0x7fbe3c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3a7fc640 1 -- 172.21.15.7:0/3307438683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1032c0 msgr2=0x7fbe3c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3a7fc640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1032c0 0x7fbe3c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3a7fc640 1 -- 172.21.15.7:0/3307438683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101710 msgr2=0x7fbe3c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3a7fc640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101710 0x7fbe3c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3a7fc640 1 -- 172.21.15.7:0/3307438683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe34009490 con 0x7fbe3c102900 2024-03-23T13:22:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe3a7fc640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c102900 0x7fbe3c197d20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400e9c0 tx=0x7fbe2400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe33fff640 1 -- 172.21.15.7:0/3307438683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2400ccc0 con 0x7fbe3c102900 2024-03-23T13:22:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe33fff640 1 -- 172.21.15.7:0/3307438683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2400ce20 con 0x7fbe3c102900 2024-03-23T13:22:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe417d6640 1 -- 172.21.15.7:0/3307438683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3c19f8e0 con 0x7fbe3c102900 2024-03-23T13:22:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe33fff640 1 -- 172.21.15.7:0/3307438683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24010430 con 0x7fbe3c102900 2024-03-23T13:22:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.948+0000 7fbe417d6640 1 -- 172.21.15.7:0/3307438683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3c19fe40 con 0x7fbe3c102900 2024-03-23T13:22:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.949+0000 7fbe417d6640 1 -- 172.21.15.7:0/3307438683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3c108a50 con 0x7fbe3c102900 2024-03-23T13:22:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.950+0000 7fbe33fff640 1 -- 172.21.15.7:0/3307438683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fbe24004f80 con 0x7fbe3c102900 2024-03-23T13:22:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.951+0000 7fbe33fff640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe280a2140 0x7fbe280a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.951+0000 7fbe3affd640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe280a2140 0x7fbe280a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.951+0000 7fbe33fff640 1 -- 172.21.15.7:0/3307438683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fbe24014070 con 0x7fbe3c102900 2024-03-23T13:22:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.952+0000 7fbe3affd640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe280a2140 0x7fbe280a4600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009830 tx=0x7fbe34034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:06.955+0000 7fbe33fff640 1 -- 172.21.15.7:0/3307438683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fbe240970f0 con 0x7fbe3c102900 2024-03-23T13:22:07.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.138+0000 7fbe417d6640 1 -- 172.21.15.7:0/3307438683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbe3c10d310 con 0x7fbe3c102900 2024-03-23T13:22:07.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.139+0000 7fbe33fff640 1 -- 172.21.15.7:0/3307438683 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fbe2409a7b0 con 0x7fbe3c102900 2024-03-23T13:22:07.143 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-23T13:22:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 -- 172.21.15.7:0/3307438683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe280a2140 msgr2=0x7fbe280a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe280a2140 0x7fbe280a4600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009830 tx=0x7fbe34034c00 comp rx=0 tx=0).stop 2024-03-23T13:22:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 -- 172.21.15.7:0/3307438683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c102900 msgr2=0x7fbe3c197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c102900 0x7fbe3c197d20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400e9c0 tx=0x7fbe2400ee80 comp rx=0 tx=0).stop 2024-03-23T13:22:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 -- 172.21.15.7:0/3307438683 shutdown_connections 2024-03-23T13:22:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fbe280a2140 0x7fbe280a4600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c1032c0 0x7fbe3c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c102900 0x7fbe3c197d20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 --2- 172.21.15.7:0/3307438683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c101710 0x7fbe3c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 -- 172.21.15.7:0/3307438683 >> 172.21.15.7:0/3307438683 conn(0x7fbe3c0fcc80 msgr2=0x7fbe3c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 -- 172.21.15.7:0/3307438683 shutdown_connections 2024-03-23T13:22:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.141+0000 7fbe417d6640 1 -- 172.21.15.7:0/3307438683 wait complete. 2024-03-23T13:22:07.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T13:22:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.256+0000 7f929abf4640 1 -- 172.21.15.7:0/425329298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92941021c0 msgr2=0x7f9294108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.256+0000 7f929abf4640 1 --2- 172.21.15.7:0/425329298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92941021c0 0x7f9294108a50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f928c009ca0 tx=0x7f928c02c0f0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.256+0000 7f929abf4640 1 -- 172.21.15.7:0/425329298 shutdown_connections 2024-03-23T13:22:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.256+0000 7f929abf4640 1 --2- 172.21.15.7:0/425329298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92941021c0 0x7f9294108a50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.256+0000 7f929abf4640 1 --2- 172.21.15.7:0/425329298 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9294101800 0x7f9294101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.256+0000 7f929abf4640 1 --2- 172.21.15.7:0/425329298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9294100610 0x7f9294100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.256+0000 7f929abf4640 1 -- 172.21.15.7:0/425329298 >> 172.21.15.7:0/425329298 conn(0x7f92940fbb80 msgr2=0x7f92940fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.256+0000 7f929abf4640 1 -- 172.21.15.7:0/425329298 shutdown_connections 2024-03-23T13:22:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.256+0000 7f929abf4640 1 -- 172.21.15.7:0/425329298 wait complete. 2024-03-23T13:22:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f929abf4640 1 Processor -- start 2024-03-23T13:22:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f929abf4640 1 -- start start 2024-03-23T13:22:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f929abf4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9294100610 0x7f92941045a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f929abf4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9294101800 0x7f9294102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f929abf4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92941021c0 0x7f9294103120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f929abf4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9294103880 con 0x7f92941021c0 2024-03-23T13:22:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f929abf4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f92941039f0 con 0x7f9294100610 2024-03-23T13:22:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f929abf4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f92941a17f0 con 0x7f9294101800 2024-03-23T13:22:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9299191640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92941021c0 0x7f9294103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9299191640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92941021c0 0x7f9294103120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34116/0 (socket says 172.21.15.7:34116) 2024-03-23T13:22:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9299191640 1 -- 172.21.15.7:0/1909786949 learned_addr learned my addr 172.21.15.7:0/1909786949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9293fff640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9294101800 0x7f9294102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9298990640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9294100610 0x7f92941045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9299191640 1 -- 172.21.15.7:0/1909786949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9294101800 msgr2=0x7f9294102be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9299191640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9294101800 0x7f9294102be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9299191640 1 -- 172.21.15.7:0/1909786949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9294100610 msgr2=0x7f92941045a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9299191640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9294100610 0x7f92941045a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9299191640 1 -- 172.21.15.7:0/1909786949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c009900 con 0x7f92941021c0 2024-03-23T13:22:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9299191640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92941021c0 0x7f9294103120 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f928c033000 tx=0x7f928c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.257+0000 7f9291ffb640 1 -- 172.21.15.7:0/1909786949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c0329b0 con 0x7f92941021c0 2024-03-23T13:22:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.258+0000 7f9291ffb640 1 -- 172.21.15.7:0/1909786949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f928c003e50 con 0x7f92941021c0 2024-03-23T13:22:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.258+0000 7f929abf4640 1 -- 172.21.15.7:0/1909786949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92941a1a40 con 0x7f92941021c0 2024-03-23T13:22:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.258+0000 7f9291ffb640 1 -- 172.21.15.7:0/1909786949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c040450 con 0x7f92941021c0 2024-03-23T13:22:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.258+0000 7f929abf4640 1 -- 172.21.15.7:0/1909786949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92941a1f40 con 0x7f92941021c0 2024-03-23T13:22:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.259+0000 7f929abf4640 1 -- 172.21.15.7:0/1909786949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f929410c020 con 0x7f92941021c0 2024-03-23T13:22:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.260+0000 7f9291ffb640 1 -- 172.21.15.7:0/1909786949 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f928c00a040 con 0x7f92941021c0 2024-03-23T13:22:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.260+0000 7f9291ffb640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92840a2140 0x7f92840a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.260+0000 7f9298990640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92840a2140 0x7f92840a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.260+0000 7f9291ffb640 1 -- 172.21.15.7:0/1909786949 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f928c0f0e70 con 0x7f92941021c0 2024-03-23T13:22:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.261+0000 7f9298990640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92840a2140 0x7f92840a4600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9294101660 tx=0x7f9288005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.264+0000 7f9291ffb640 1 -- 172.21.15.7:0/1909786949 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f928c0b9ff0 con 0x7f92941021c0 2024-03-23T13:22:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.460+0000 7f929abf4640 1 -- 172.21.15.7:0/1909786949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f92941a2130 con 0x7f92941021c0 2024-03-23T13:22:07.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.461+0000 7f9291ffb640 1 -- 172.21.15.7:0/1909786949 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+579 (secure 0 0 0) 0x7f928c039260 con 0x7f92941021c0 2024-03-23T13:22:07.465 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2507019439 2024-03-24T13:10:41.843648+0000 2024-03-23T13:22:07.465 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1286669735 2024-03-24T13:10:41.843648+0000 2024-03-23T13:22:07.465 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3684792738 2024-03-24T13:10:41.843648+0000 2024-03-23T13:22:07.466 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3945637763 2024-03-24T13:10:41.843648+0000 2024-03-23T13:22:07.466 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/742242630 2024-03-24T13:10:15.062657+0000 2024-03-23T13:22:07.466 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/310337452 2024-03-24T13:10:15.062657+0000 2024-03-23T13:22:07.466 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/310337452 2024-03-24T13:10:15.062657+0000 2024-03-23T13:22:07.466 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1496728145 2024-03-24T13:10:15.062657+0000 2024-03-23T13:22:07.466 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3945637763 2024-03-24T13:10:41.843648+0000 2024-03-23T13:22:07.466 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1695133666 2024-03-24T13:10:15.062657+0000 2024-03-23T13:22:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-03-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 -- 172.21.15.7:0/1909786949 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92840a2140 msgr2=0x7f92840a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92840a2140 0x7f92840a4600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9294101660 tx=0x7f9288005f90 comp rx=0 tx=0).stop 2024-03-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 -- 172.21.15.7:0/1909786949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92941021c0 msgr2=0x7f9294103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92941021c0 0x7f9294103120 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f928c033000 tx=0x7f928c009360 comp rx=0 tx=0).stop 2024-03-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 -- 172.21.15.7:0/1909786949 shutdown_connections 2024-03-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f92840a2140 0x7f92840a4600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92941021c0 0x7f9294103120 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9294101800 0x7f9294102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 --2- 172.21.15.7:0/1909786949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9294100610 0x7f92941045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 -- 172.21.15.7:0/1909786949 >> 172.21.15.7:0/1909786949 conn(0x7f92940fbb80 msgr2=0x7f92940fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 -- 172.21.15.7:0/1909786949 shutdown_connections 2024-03-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.463+0000 7f929abf4640 1 -- 172.21.15.7:0/1909786949 wait complete. 2024-03-23T13:22:07.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:07 smithi007 ceph-mon[32184]: pgmap v419: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 15 op/s 2024-03-23T13:22:07.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:07 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1806470678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:22:07.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:07 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1101743027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:22:07.478 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:07 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3307438683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:22:07.479 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-23T13:22:07.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:22:07.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-23T13:22:07.561 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-03-23T13:22:07.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:22:07.561 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T13:22:07.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:22:07.597 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-23T13:22:07.661 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T13:22:07.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:22:07.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T13:22:07.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:07 smithi130 ceph-mon[36225]: pgmap v419: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 15 op/s 2024-03-23T13:22:07.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:07 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1806470678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:22:07.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:07 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1101743027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:22:07.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:07 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3307438683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:22:07.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T13:22:07.740 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-03-23T13:22:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:07 smithi155 ceph-mon[36157]: pgmap v419: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 15 op/s 2024-03-23T13:22:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:07 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1806470678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T13:22:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:07 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1101743027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T13:22:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:07 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3307438683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T13:22:07.836 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-23T13:22:07.846 INFO:tasks.cephadm:Teardown begin 2024-03-23T13:22:07.847 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T13:22:07.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:22:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.947+0000 7f82cc40e640 1 -- 172.21.15.7:0/3385302019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4100630 msgr2=0x7f82c4100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.947+0000 7f82cc40e640 1 --2- 172.21.15.7:0/3385302019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4100630 0x7f82c4100a50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009900 tx=0x7f82bc02bc30 comp rx=0 tx=0).stop 2024-03-23T13:22:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.947+0000 7f82cc40e640 1 -- 172.21.15.7:0/3385302019 shutdown_connections 2024-03-23T13:22:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.947+0000 7f82cc40e640 1 --2- 172.21.15.7:0/3385302019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c41021e0 0x7f82c4108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.947+0000 7f82cc40e640 1 --2- 172.21.15.7:0/3385302019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c4101820 0x7f82c4101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.947+0000 7f82cc40e640 1 --2- 172.21.15.7:0/3385302019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4100630 0x7f82c4100a50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.947+0000 7f82cc40e640 1 -- 172.21.15.7:0/3385302019 >> 172.21.15.7:0/3385302019 conn(0x7f82c40fbba0 msgr2=0x7f82c40fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82cc40e640 1 -- 172.21.15.7:0/3385302019 shutdown_connections 2024-03-23T13:22:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82cc40e640 1 -- 172.21.15.7:0/3385302019 wait complete. 2024-03-23T13:22:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82cc40e640 1 Processor -- start 2024-03-23T13:22:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82cc40e640 1 -- start start 2024-03-23T13:22:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82cc40e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4100630 0x7f82c41045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82cc40e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4101820 0x7f82c4102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82cc40e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c41021e0 0x7f82c4103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82cc40e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41038a0 con 0x7f82c4100630 2024-03-23T13:22:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82cc40e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4103a10 con 0x7f82c41021e0 2024-03-23T13:22:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82cc40e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41a18b0 con 0x7f82c4101820 2024-03-23T13:22:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82ca1aa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4100630 0x7f82c41045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82ca1aa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4100630 0x7f82c41045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34146/0 (socket says 172.21.15.7:34146) 2024-03-23T13:22:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82ca1aa640 1 -- 172.21.15.7:0/2658452205 learned_addr learned my addr 172.21.15.7:0/2658452205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82c99a9640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4101820 0x7f82c4102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.948+0000 7f82ca9ab640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c41021e0 0x7f82c4103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82ca1aa640 1 -- 172.21.15.7:0/2658452205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4101820 msgr2=0x7f82c4102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82ca1aa640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4101820 0x7f82c4102c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82ca1aa640 1 -- 172.21.15.7:0/2658452205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c41021e0 msgr2=0x7f82c4103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82ca1aa640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c41021e0 0x7f82c4103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82ca1aa640 1 -- 172.21.15.7:0/2658452205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc009560 con 0x7f82c4100630 2024-03-23T13:22:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82ca9ab640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c41021e0 0x7f82c4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:22:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82ca1aa640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4100630 0x7f82c41045c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f82bc036040 tx=0x7f82bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82c37fe640 1 -- 172.21.15.7:0/2658452205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc004df0 con 0x7f82c4100630 2024-03-23T13:22:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82c37fe640 1 -- 172.21.15.7:0/2658452205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc005430 con 0x7f82c4100630 2024-03-23T13:22:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82cc40e640 1 -- 172.21.15.7:0/2658452205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c41a1b00 con 0x7f82c4100630 2024-03-23T13:22:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82c37fe640 1 -- 172.21.15.7:0/2658452205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc03e450 con 0x7f82c4100630 2024-03-23T13:22:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.949+0000 7f82cc40e640 1 -- 172.21.15.7:0/2658452205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a2060 con 0x7f82c4100630 2024-03-23T13:22:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.950+0000 7f82cc40e640 1 -- 172.21.15.7:0/2658452205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c410c040 con 0x7f82c4100630 2024-03-23T13:22:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.951+0000 7f82c37fe640 1 -- 172.21.15.7:0/2658452205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f82bc003680 con 0x7f82c4100630 2024-03-23T13:22:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.951+0000 7f82c37fe640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f82ac0a2140 0x7f82ac0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.952+0000 7f82c99a9640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f82ac0a2140 0x7f82ac0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.952+0000 7f82c37fe640 1 -- 172.21.15.7:0/2658452205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f82bc0f0740 con 0x7f82c4100630 2024-03-23T13:22:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.952+0000 7f82c99a9640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f82ac0a2140 0x7f82ac0a4600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82c4104330 tx=0x7f82b800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:07.956+0000 7f82c37fe640 1 -- 172.21.15.7:0/2658452205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f82bc0b9720 con 0x7f82c4100630 2024-03-23T13:22:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.175+0000 7f82cc40e640 1 -- 172.21.15.7:0/2658452205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82c410c1d0 con 0x7f82c4100630 2024-03-23T13:22:08.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.176+0000 7f82c37fe640 1 -- 172.21.15.7:0/2658452205 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11924 (secure 0 0 0) 0x7f82bc0bcde0 con 0x7f82c4100630 2024-03-23T13:22:08.179 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:22:08.180 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T13:17:40.597309+0000","modified":"2024-03-23T13:20:52.027317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24475,"mds_3":24470,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6841/3415570481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3415570481},{"type":"v1","addr":"172.21.15.7:6841","nonce":3415570481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/791500055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":791500055},{"type":"v1","addr":"172.21.15.130:6839","nonce":791500055}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24470,24463,24475,24503,24496]},"id":1}]} 2024-03-23T13:22:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-03-23T13:22:08.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 -- 172.21.15.7:0/2658452205 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f82ac0a2140 msgr2=0x7f82ac0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f82ac0a2140 0x7f82ac0a4600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82c4104330 tx=0x7f82b800b390 comp rx=0 tx=0).stop 2024-03-23T13:22:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 -- 172.21.15.7:0/2658452205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4100630 msgr2=0x7f82c41045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4100630 0x7f82c41045c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f82bc036040 tx=0x7f82bc004270 comp rx=0 tx=0).stop 2024-03-23T13:22:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 -- 172.21.15.7:0/2658452205 shutdown_connections 2024-03-23T13:22:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f82ac0a2140 0x7f82ac0a4600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c41021e0 0x7f82c4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4101820 0x7f82c4102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 --2- 172.21.15.7:0/2658452205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4100630 0x7f82c41045c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 -- 172.21.15.7:0/2658452205 >> 172.21.15.7:0/2658452205 conn(0x7f82c40fbba0 msgr2=0x7f82c4068960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:08.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 -- 172.21.15.7:0/2658452205 shutdown_connections 2024-03-23T13:22:08.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.178+0000 7f82cc40e640 1 -- 172.21.15.7:0/2658452205 wait complete. 2024-03-23T13:22:08.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T13:22:08.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.291+0000 7f1b81fe2640 1 -- 172.21.15.7:0/659734280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b7c100630 msgr2=0x7f1b7c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.291+0000 7f1b81fe2640 1 --2- 172.21.15.7:0/659734280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b7c100630 0x7f1b7c100a50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009900 tx=0x7f1b6c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:22:08.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 -- 172.21.15.7:0/659734280 shutdown_connections 2024-03-23T13:22:08.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 --2- 172.21.15.7:0/659734280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c1021e0 0x7f1b7c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 --2- 172.21.15.7:0/659734280 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b7c101820 0x7f1b7c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 --2- 172.21.15.7:0/659734280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b7c100630 0x7f1b7c100a50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 -- 172.21.15.7:0/659734280 >> 172.21.15.7:0/659734280 conn(0x7f1b7c0fbba0 msgr2=0x7f1b7c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:08.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 -- 172.21.15.7:0/659734280 shutdown_connections 2024-03-23T13:22:08.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 -- 172.21.15.7:0/659734280 wait complete. 2024-03-23T13:22:08.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 Processor -- start 2024-03-23T13:22:08.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 -- start start 2024-03-23T13:22:08.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c100630 0x7f1b7c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b7c101820 0x7f1b7c197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b7c1021e0 0x7f1b7c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c19f2d0 con 0x7f1b7c1021e0 2024-03-23T13:22:08.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c19f440 con 0x7f1b7c101820 2024-03-23T13:22:08.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.292+0000 7f1b81fe2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c19f600 con 0x7f1b7c100630 2024-03-23T13:22:08.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b7c1021e0 0x7f1b7c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b7c1021e0 0x7f1b7c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34168/0 (socket says 172.21.15.7:34168) 2024-03-23T13:22:08.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7bfff640 1 -- 172.21.15.7:0/4108718222 learned_addr learned my addr 172.21.15.7:0/4108718222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:08.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7b7fe640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c100630 0x7f1b7c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7affd640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b7c101820 0x7f1b7c197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7bfff640 1 -- 172.21.15.7:0/4108718222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c100630 msgr2=0x7f1b7c197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7bfff640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c100630 0x7f1b7c197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7bfff640 1 -- 172.21.15.7:0/4108718222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b7c101820 msgr2=0x7f1b7c197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7bfff640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b7c101820 0x7f1b7c197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7bfff640 1 -- 172.21.15.7:0/4108718222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c009560 con 0x7f1b7c1021e0 2024-03-23T13:22:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b7bfff640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b7c1021e0 0x7f1b7c19ed90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400eb10 tx=0x7f1b7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b78ff9640 1 -- 172.21.15.7:0/4108718222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74004b20 con 0x7f1b7c1021e0 2024-03-23T13:22:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b81fe2640 1 -- 172.21.15.7:0/4108718222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c19f8e0 con 0x7f1b7c1021e0 2024-03-23T13:22:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b78ff9640 1 -- 172.21.15.7:0/4108718222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b74004c80 con 0x7f1b7c1021e0 2024-03-23T13:22:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b78ff9640 1 -- 172.21.15.7:0/4108718222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74010670 con 0x7f1b7c1021e0 2024-03-23T13:22:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.293+0000 7f1b81fe2640 1 -- 172.21.15.7:0/4108718222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c19fe40 con 0x7f1b7c1021e0 2024-03-23T13:22:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.294+0000 7f1b81fe2640 1 -- 172.21.15.7:0/4108718222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c10c040 con 0x7f1b7c1021e0 2024-03-23T13:22:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.295+0000 7f1b78ff9640 1 -- 172.21.15.7:0/4108718222 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f1b74005020 con 0x7f1b7c1021e0 2024-03-23T13:22:08.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.296+0000 7f1b78ff9640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1b640a20c0 0x7f1b640a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.296+0000 7f1b78ff9640 1 -- 172.21.15.7:0/4108718222 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1b74014070 con 0x7f1b7c1021e0 2024-03-23T13:22:08.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.296+0000 7f1b7b7fe640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1b640a20c0 0x7f1b640a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.296+0000 7f1b7b7fe640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1b640a20c0 0x7f1b640a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c005ab0 tx=0x7f1b6c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.300+0000 7f1b78ff9640 1 -- 172.21.15.7:0/4108718222 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1b740971c0 con 0x7f1b7c1021e0 2024-03-23T13:22:08.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.524+0000 7f1b81fe2640 1 -- 172.21.15.7:0/4108718222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b7c1a00f0 con 0x7f1b7c1021e0 2024-03-23T13:22:08.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.525+0000 7f1b78ff9640 1 -- 172.21.15.7:0/4108718222 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11924 (secure 0 0 0) 0x7f1b7409a880 con 0x7f1b7c1021e0 2024-03-23T13:22:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:22:08.528 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3229137818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3229137818},{"type":"v1","addr":"172.21.15.7:6835","nonce":3229137818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/746475377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":746475377},{"type":"v1","addr":"172.21.15.7:6837","nonce":746475377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T13:17:40.597309+0000","modified":"2024-03-23T13:20:52.027317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24475,"mds_3":24470,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6839/388542752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":388542752},{"type":"v1","addr":"172.21.15.7:6839","nonce":388542752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6841/3415570481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3415570481},{"type":"v1","addr":"172.21.15.7:6841","nonce":3415570481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6833/3818242586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":3818242586},{"type":"v1","addr":"172.21.15.130:6833","nonce":3818242586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1319723529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1319723529},{"type":"v1","addr":"172.21.15.155:6833","nonce":1319723529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2599360099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2599360099},{"type":"v1","addr":"172.21.15.130:6835","nonce":2599360099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/1880391142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1880391142},{"type":"v1","addr":"172.21.15.155:6835","nonce":1880391142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.130:6837/877142007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":877142007},{"type":"v1","addr":"172.21.15.130:6837","nonce":877142007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.155:6837/3999767587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3999767587},{"type":"v1","addr":"172.21.15.155:6837","nonce":3999767587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6839/131946068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":131946068},{"type":"v1","addr":"172.21.15.155:6839","nonce":131946068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.130:6839/791500055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":791500055},{"type":"v1","addr":"172.21.15.130:6839","nonce":791500055}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24470,24463,24475,24503,24496]},"id":1}]} 2024-03-23T13:22:08.529 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-03-23T13:22:08.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 -- 172.21.15.7:0/4108718222 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1b640a20c0 msgr2=0x7f1b640a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1b640a20c0 0x7f1b640a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c005ab0 tx=0x7f1b6c005fd0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 -- 172.21.15.7:0/4108718222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b7c1021e0 msgr2=0x7f1b7c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b7c1021e0 0x7f1b7c19ed90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400eb10 tx=0x7f1b7400c2d0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 -- 172.21.15.7:0/4108718222 shutdown_connections 2024-03-23T13:22:08.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f1b640a20c0 0x7f1b640a4580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b7c1021e0 0x7f1b7c19ed90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b7c101820 0x7f1b7c197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 --2- 172.21.15.7:0/4108718222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c100630 0x7f1b7c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 -- 172.21.15.7:0/4108718222 >> 172.21.15.7:0/4108718222 conn(0x7f1b7c0fbba0 msgr2=0x7f1b7c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:08.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 -- 172.21.15.7:0/4108718222 shutdown_connections 2024-03-23T13:22:08.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.528+0000 7f1b81fe2640 1 -- 172.21.15.7:0/4108718222 wait complete. 2024-03-23T13:22:08.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:08 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1909786949' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T13:22:08.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:08 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2658452205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:22:08.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T13:22:08.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 -- 172.21.15.7:0/2518407547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef5c102260 msgr2=0x7fef5c108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 --2- 172.21.15.7:0/2518407547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef5c102260 0x7fef5c108af0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fef54009ca0 tx=0x7fef5402c0f0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 -- 172.21.15.7:0/2518407547 shutdown_connections 2024-03-23T13:22:08.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 --2- 172.21.15.7:0/2518407547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef5c102260 0x7fef5c108af0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 --2- 172.21.15.7:0/2518407547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef5c1018a0 0x7fef5c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 --2- 172.21.15.7:0/2518407547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef5c1006b0 0x7fef5c100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 -- 172.21.15.7:0/2518407547 >> 172.21.15.7:0/2518407547 conn(0x7fef5c0fbc40 msgr2=0x7fef5c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:08.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 -- 172.21.15.7:0/2518407547 shutdown_connections 2024-03-23T13:22:08.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 -- 172.21.15.7:0/2518407547 wait complete. 2024-03-23T13:22:08.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 Processor -- start 2024-03-23T13:22:08.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.644+0000 7fef60f9a640 1 -- start start 2024-03-23T13:22:08.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef60f9a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef5c1006b0 0x7fef5c1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef60f9a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef5c1018a0 0x7fef5c196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef60f9a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef5c102260 0x7fef5c19dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef60f9a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c19e190 con 0x7fef5c1018a0 2024-03-23T13:22:08.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef60f9a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c19e300 con 0x7fef5c1006b0 2024-03-23T13:22:08.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef60f9a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c19e4c0 con 0x7fef5c102260 2024-03-23T13:22:08.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef53fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef5c1018a0 0x7fef5c196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef53fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef5c1018a0 0x7fef5c196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34198/0 (socket says 172.21.15.7:34198) 2024-03-23T13:22:08.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef53fff640 1 -- 172.21.15.7:0/2986857227 learned_addr learned my addr 172.21.15.7:0/2986857227 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:08.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef5a59c640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef5c1006b0 0x7fef5c1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef5ad9d640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef5c102260 0x7fef5c19dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef53fff640 1 -- 172.21.15.7:0/2986857227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef5c102260 msgr2=0x7fef5c19dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef53fff640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef5c102260 0x7fef5c19dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef53fff640 1 -- 172.21.15.7:0/2986857227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef5c1006b0 msgr2=0x7fef5c1966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef53fff640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef5c1006b0 0x7fef5c1966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef53fff640 1 -- 172.21.15.7:0/2986857227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef54009900 con 0x7fef5c1018a0 2024-03-23T13:22:08.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef53fff640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef5c1018a0 0x7fef5c196be0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fef4400e9f0 tx=0x7fef4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:08.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef537fe640 1 -- 172.21.15.7:0/2986857227 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44009720 con 0x7fef5c1018a0 2024-03-23T13:22:08.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef537fe640 1 -- 172.21.15.7:0/2986857227 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef4400cd80 con 0x7fef5c1018a0 2024-03-23T13:22:08.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef60f9a640 1 -- 172.21.15.7:0/2986857227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c19e7a0 con 0x7fef5c1018a0 2024-03-23T13:22:08.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef537fe640 1 -- 172.21.15.7:0/2986857227 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44010690 con 0x7fef5c1018a0 2024-03-23T13:22:08.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.645+0000 7fef60f9a640 1 -- 172.21.15.7:0/2986857227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c19ec60 con 0x7fef5c1018a0 2024-03-23T13:22:08.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.646+0000 7fef60f9a640 1 -- 172.21.15.7:0/2986857227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c1079d0 con 0x7fef5c1018a0 2024-03-23T13:22:08.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.648+0000 7fef537fe640 1 -- 172.21.15.7:0/2986857227 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fef440046c0 con 0x7fef5c1018a0 2024-03-23T13:22:08.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.648+0000 7fef537fe640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef480a20c0 0x7fef480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.648+0000 7fef5a59c640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef480a20c0 0x7fef480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.648+0000 7fef537fe640 1 -- 172.21.15.7:0/2986857227 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fef44014070 con 0x7fef5c1018a0 2024-03-23T13:22:08.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.649+0000 7fef5a59c640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef480a20c0 0x7fef480a4580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef5c101700 tx=0x7fef4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:08.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.652+0000 7fef537fe640 1 -- 172.21.15.7:0/2986857227 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fef44097200 con 0x7fef5c1018a0 2024-03-23T13:22:08.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:08 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1909786949' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T13:22:08.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:08 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2658452205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:22:08.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:08 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1909786949' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T13:22:08.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:08 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2658452205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:22:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.835+0000 7fef60f9a640 1 -- 172.21.15.7:0/2986857227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef5c10c050 con 0x7fef5c1018a0 2024-03-23T13:22:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.837+0000 7fef537fe640 1 -- 172.21.15.7:0/2986857227 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22558 (secure 0 0 0) 0x7fef4409a8c0 con 0x7fef5c1018a0 2024-03-23T13:22:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-23T13:22:08.840 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"745f591a-e916-11ee-95cd-87774f69a715","created":"2024-03-23T13:09:39.213712+0000","modified":"2024-03-23T13:18:43.667397+0000","last_up_change":"2024-03-23T13:16:53.557012+0000","last_in_change":"2024-03-23T13:16:33.517021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T13:13:08.186719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T13:17:38.843887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T13:17:39.788489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0da10b6e-bba6-44c4-ae11-5378f134eb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6803","nonce":2113068565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6805","nonce":2113068565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6809","nonce":2113068565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2113068565},{"type":"v1","addr":"172.21.15.7:6807","nonce":2113068565}]},"public_addr":"172.21.15.7:6803/2113068565","cluster_addr":"172.21.15.7:6805/2113068565","heartbeat_back_addr":"172.21.15.7:6809/2113068565","heartbeat_front_addr":"172.21.15.7:6807/2113068565","state":["exists","up"]},{"osd":1,"uuid":"52bc95ae-72ec-43dd-b7b7-7a88b6593c0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6801","nonce":704785998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6803","nonce":704785998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6807","nonce":704785998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":704785998},{"type":"v1","addr":"172.21.15.130:6805","nonce":704785998}]},"public_addr":"172.21.15.130:6801/704785998","cluster_addr":"172.21.15.130:6803/704785998","heartbeat_back_addr":"172.21.15.130:6807/704785998","heartbeat_front_addr":"172.21.15.130:6805/704785998","state":["exists","up"]},{"osd":2,"uuid":"2013fd43-79f2-420b-8518-046fce065cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6801","nonce":2256065708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6803","nonce":2256065708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6807","nonce":2256065708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2256065708},{"type":"v1","addr":"172.21.15.155:6805","nonce":2256065708}]},"public_addr":"172.21.15.155:6801/2256065708","cluster_addr":"172.21.15.155:6803/2256065708","heartbeat_back_addr":"172.21.15.155:6807/2256065708","heartbeat_front_addr":"172.21.15.155:6805/2256065708","state":["exists","up"]},{"osd":3,"uuid":"1a418507-c9a0-4945-aca6-43c156f4f96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6811","nonce":28573179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6813","nonce":28573179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6817","nonce":28573179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":28573179},{"type":"v1","addr":"172.21.15.7:6815","nonce":28573179}]},"public_addr":"172.21.15.7:6811/28573179","cluster_addr":"172.21.15.7:6813/28573179","heartbeat_back_addr":"172.21.15.7:6817/28573179","heartbeat_front_addr":"172.21.15.7:6815/28573179","state":["exists","up"]},{"osd":4,"uuid":"258432a8-3995-4878-928f-1bc7cc486699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6809","nonce":568360270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6811","nonce":568360270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6815","nonce":568360270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":568360270},{"type":"v1","addr":"172.21.15.130:6813","nonce":568360270}]},"public_addr":"172.21.15.130:6809/568360270","cluster_addr":"172.21.15.130:6811/568360270","heartbeat_back_addr":"172.21.15.130:6815/568360270","heartbeat_front_addr":"172.21.15.130:6813/568360270","state":["exists","up"]},{"osd":5,"uuid":"f115a6d4-3043-48d6-a29e-b367c62dd418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6809","nonce":3878988779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6811","nonce":3878988779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6815","nonce":3878988779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3878988779},{"type":"v1","addr":"172.21.15.155:6813","nonce":3878988779}]},"public_addr":"172.21.15.155:6809/3878988779","cluster_addr":"172.21.15.155:6811/3878988779","heartbeat_back_addr":"172.21.15.155:6815/3878988779","heartbeat_front_addr":"172.21.15.155:6813/3878988779","state":["exists","up"]},{"osd":6,"uuid":"16353022-32db-487a-99e8-d8fe28fdaef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6819","nonce":2961397524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6821","nonce":2961397524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6825","nonce":2961397524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2961397524},{"type":"v1","addr":"172.21.15.7:6823","nonce":2961397524}]},"public_addr":"172.21.15.7:6819/2961397524","cluster_addr":"172.21.15.7:6821/2961397524","heartbeat_back_addr":"172.21.15.7:6825/2961397524","heartbeat_front_addr":"172.21.15.7:6823/2961397524","state":["exists","up"]},{"osd":7,"uuid":"c3b2652e-e404-4893-aff0-5049956327ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6817","nonce":869404947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6819","nonce":869404947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6823","nonce":869404947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":869404947},{"type":"v1","addr":"172.21.15.130:6821","nonce":869404947}]},"public_addr":"172.21.15.130:6817/869404947","cluster_addr":"172.21.15.130:6819/869404947","heartbeat_back_addr":"172.21.15.130:6823/869404947","heartbeat_front_addr":"172.21.15.130:6821/869404947","state":["exists","up"]},{"osd":8,"uuid":"46014832-6a35-46b2-a918-ef9179dc3bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6817","nonce":2565353660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6819","nonce":2565353660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6823","nonce":2565353660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2565353660},{"type":"v1","addr":"172.21.15.155:6821","nonce":2565353660}]},"public_addr":"172.21.15.155:6817/2565353660","cluster_addr":"172.21.15.155:6819/2565353660","heartbeat_back_addr":"172.21.15.155:6823/2565353660","heartbeat_front_addr":"172.21.15.155:6821/2565353660","state":["exists","up"]},{"osd":9,"uuid":"6fdeaf21-f0a5-4dff-ac58-8bb059972ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6827","nonce":439190456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6829","nonce":439190456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6833","nonce":439190456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":439190456},{"type":"v1","addr":"172.21.15.7:6831","nonce":439190456}]},"public_addr":"172.21.15.7:6827/439190456","cluster_addr":"172.21.15.7:6829/439190456","heartbeat_back_addr":"172.21.15.7:6833/439190456","heartbeat_front_addr":"172.21.15.7:6831/439190456","state":["exists","up"]},{"osd":10,"uuid":"06ebd1ee-dd0b-424f-9e4d-113d31950c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6825","nonce":1468057097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6827","nonce":1468057097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6831","nonce":1468057097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1468057097},{"type":"v1","addr":"172.21.15.130:6829","nonce":1468057097}]},"public_addr":"172.21.15.130:6825/1468057097","cluster_addr":"172.21.15.130:6827/1468057097","heartbeat_back_addr":"172.21.15.130:6831/1468057097","heartbeat_front_addr":"172.21.15.130:6829/1468057097","state":["exists","up"]},{"osd":11,"uuid":"fc34a0d3-024e-4b1f-907b-147a3fd1298d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6825","nonce":2887615966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6827","nonce":2887615966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6831","nonce":2887615966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2887615966},{"type":"v1","addr":"172.21.15.155:6829","nonce":2887615966}]},"public_addr":"172.21.15.155:6825/2887615966","cluster_addr":"172.21.15.155:6827/2887615966","heartbeat_back_addr":"172.21.15.155:6831/2887615966","heartbeat_front_addr":"172.21.15.155:6829/2887615966","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:16.547546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:12:40.023309+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:03.910747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:28.622884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:13:53.363439+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:18.112406+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:14:43.364423+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:08.702647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:34.257518+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:15:59.878688+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:25.860978+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T13:16:51.862879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/2507019439":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1286669735":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/3684792738":"2024-03-24T13:10:41.843648+0000","172.21.15.7:6801/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/742242630":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6801/310337452":"2024-03-24T13:10:15.062657+0000","172.21.15.7:0/1496728145":"2024-03-24T13:10:15.062657+0000","172.21.15.7:6800/3945637763":"2024-03-24T13:10:41.843648+0000","172.21.15.7:0/1695133666":"2024-03-24T13:10:15.062657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T13:22:08.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 -- 172.21.15.7:0/2986857227 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef480a20c0 msgr2=0x7fef480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef480a20c0 0x7fef480a4580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef5c101700 tx=0x7fef4c005f90 comp rx=0 tx=0).stop 2024-03-23T13:22:08.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 -- 172.21.15.7:0/2986857227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef5c1018a0 msgr2=0x7fef5c196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef5c1018a0 0x7fef5c196be0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fef4400e9f0 tx=0x7fef4400edf0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 -- 172.21.15.7:0/2986857227 shutdown_connections 2024-03-23T13:22:08.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fef480a20c0 0x7fef480a4580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef5c102260 0x7fef5c19dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef5c1018a0 0x7fef5c196be0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 --2- 172.21.15.7:0/2986857227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef5c1006b0 0x7fef5c1966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 -- 172.21.15.7:0/2986857227 >> 172.21.15.7:0/2986857227 conn(0x7fef5c0fbc40 msgr2=0x7fef5c104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 -- 172.21.15.7:0/2986857227 shutdown_connections 2024-03-23T13:22:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.839+0000 7fef60f9a640 1 -- 172.21.15.7:0/2986857227 wait complete. 2024-03-23T13:22:08.853 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T13:22:08.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.954+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/3165519487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9b41032d0 msgr2=0x7fb9b4109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.954+0000 7fb9bb1f6640 1 --2- 172.21.15.7:0/3165519487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9b41032d0 0x7fb9b4109b60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009ca0 tx=0x7fb9ac02c100 comp rx=0 tx=0).stop 2024-03-23T13:22:08.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.954+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/3165519487 shutdown_connections 2024-03-23T13:22:08.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.954+0000 7fb9bb1f6640 1 --2- 172.21.15.7:0/3165519487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9b41032d0 0x7fb9b4109b60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.954+0000 7fb9bb1f6640 1 --2- 172.21.15.7:0/3165519487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b4102910 0x7fb9b4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.954+0000 7fb9bb1f6640 1 --2- 172.21.15.7:0/3165519487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9b4101720 0x7fb9b4101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.954+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/3165519487 >> 172.21.15.7:0/3165519487 conn(0x7fb9b40fccb0 msgr2=0x7fb9b40ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:08.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.954+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/3165519487 shutdown_connections 2024-03-23T13:22:08.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.954+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/3165519487 wait complete. 2024-03-23T13:22:08.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9bb1f6640 1 Processor -- start 2024-03-23T13:22:08.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9bb1f6640 1 -- start start 2024-03-23T13:22:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9bb1f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b4101720 0x7fb9b4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9bb1f6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9b4102910 0x7fb9b4197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9bb1f6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9b41032d0 0x7fb9b419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9bb1f6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b419f250 con 0x7fb9b4101720 2024-03-23T13:22:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9bb1f6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b419f3c0 con 0x7fb9b41032d0 2024-03-23T13:22:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9bb1f6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b419f580 con 0x7fb9b4102910 2024-03-23T13:22:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9b8f92640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b4101720 0x7fb9b4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9b8f92640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b4101720 0x7fb9b4197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34222/0 (socket says 172.21.15.7:34222) 2024-03-23T13:22:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9b8f92640 1 -- 172.21.15.7:0/1192307549 learned_addr learned my addr 172.21.15.7:0/1192307549 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:08.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9b3fff640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9b4102910 0x7fb9b4197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9b9793640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9b41032d0 0x7fb9b419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9b8f92640 1 -- 172.21.15.7:0/1192307549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9b4102910 msgr2=0x7fb9b4197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9b8f92640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9b4102910 0x7fb9b4197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9b8f92640 1 -- 172.21.15.7:0/1192307549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9b41032d0 msgr2=0x7fb9b419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:08.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9b8f92640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9b41032d0 0x7fb9b419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:08.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.955+0000 7fb9b8f92640 1 -- 172.21.15.7:0/1192307549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac009900 con 0x7fb9b4101720 2024-03-23T13:22:08.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.956+0000 7fb9b8f92640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b4101720 0x7fb9b4197760 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800d950 tx=0x7fb9a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.956+0000 7fb9b1ffb640 1 -- 172.21.15.7:0/1192307549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8009c30 con 0x7fb9b4101720 2024-03-23T13:22:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.956+0000 7fb9b1ffb640 1 -- 172.21.15.7:0/1192307549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a8009d90 con 0x7fb9b4101720 2024-03-23T13:22:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.956+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/1192307549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b419f860 con 0x7fb9b4101720 2024-03-23T13:22:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.956+0000 7fb9b1ffb640 1 -- 172.21.15.7:0/1192307549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800f720 con 0x7fb9b4101720 2024-03-23T13:22:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.956+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/1192307549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b419fd20 con 0x7fb9b4101720 2024-03-23T13:22:08.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.957+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/1192307549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b4108a40 con 0x7fb9b4101720 2024-03-23T13:22:08.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.958+0000 7fb9b1ffb640 1 -- 172.21.15.7:0/1192307549 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7fb9a8010a60 con 0x7fb9b4101720 2024-03-23T13:22:08.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.959+0000 7fb9b1ffb640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9a40a20c0 0x7fb9a40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:08.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.959+0000 7fb9b3fff640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9a40a20c0 0x7fb9a40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:08.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.959+0000 7fb9b1ffb640 1 -- 172.21.15.7:0/1192307549 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb9a80cd9f0 con 0x7fb9b4101720 2024-03-23T13:22:08.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.959+0000 7fb9b3fff640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9a40a20c0 0x7fb9a40a4580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4198c70 tx=0x7fb9a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:08.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:08.964+0000 7fb9b1ffb640 1 -- 172.21.15.7:0/1192307549 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb9a8096a20 con 0x7fb9b4101720 2024-03-23T13:22:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.186+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/1192307549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb9b402d1d0 con 0x7fb9b4101720 2024-03-23T13:22:09.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.188+0000 7fb9b1ffb640 1 -- 172.21.15.7:0/1192307549 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7fb9a809a0e0 con 0x7fb9b4101720 2024-03-23T13:22:09.191 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T13:22:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/1192307549 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9a40a20c0 msgr2=0x7fb9a40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9a40a20c0 0x7fb9a40a4580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4198c70 tx=0x7fb9a0005f90 comp rx=0 tx=0).stop 2024-03-23T13:22:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/1192307549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b4101720 msgr2=0x7fb9b4197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b4101720 0x7fb9b4197760 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800d950 tx=0x7fb9a800de10 comp rx=0 tx=0).stop 2024-03-23T13:22:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/1192307549 shutdown_connections 2024-03-23T13:22:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7fb9a40a20c0 0x7fb9a40a4580 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9b41032d0 0x7fb9b419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:09.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9b4102910 0x7fb9b4197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:09.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 --2- 172.21.15.7:0/1192307549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b4101720 0x7fb9b4197760 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:09.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/1192307549 >> 172.21.15.7:0/1192307549 conn(0x7fb9b40fccb0 msgr2=0x7fb9b4105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:09.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/1192307549 shutdown_connections 2024-03-23T13:22:09.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.192+0000 7fb9bb1f6640 1 -- 172.21.15.7:0/1192307549 wait complete. 2024-03-23T13:22:09.207 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T13:22:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.317+0000 7ff02c04e640 1 -- 172.21.15.7:0/1335201984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff024100610 msgr2=0x7ff024100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.317+0000 7ff02c04e640 1 --2- 172.21.15.7:0/1335201984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff024100610 0x7ff024100a30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff020009900 tx=0x7ff02002bc30 comp rx=0 tx=0).stop 2024-03-23T13:22:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.317+0000 7ff02c04e640 1 -- 172.21.15.7:0/1335201984 shutdown_connections 2024-03-23T13:22:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.317+0000 7ff02c04e640 1 --2- 172.21.15.7:0/1335201984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0241021c0 0x7ff024108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.317+0000 7ff02c04e640 1 --2- 172.21.15.7:0/1335201984 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff024101800 0x7ff024101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.317+0000 7ff02c04e640 1 --2- 172.21.15.7:0/1335201984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff024100610 0x7ff024100a30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.317+0000 7ff02c04e640 1 -- 172.21.15.7:0/1335201984 >> 172.21.15.7:0/1335201984 conn(0x7ff0240fbba0 msgr2=0x7ff0240fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.317+0000 7ff02c04e640 1 -- 172.21.15.7:0/1335201984 shutdown_connections 2024-03-23T13:22:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.317+0000 7ff02c04e640 1 -- 172.21.15.7:0/1335201984 wait complete. 2024-03-23T13:22:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff02c04e640 1 Processor -- start 2024-03-23T13:22:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff02c04e640 1 -- start start 2024-03-23T13:22:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff02c04e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff024100610 0x7ff0240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff02c04e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff024101800 0x7ff02419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff02c04e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0241021c0 0x7ff02419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff02c04e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff02419f340 con 0x7ff024100610 2024-03-23T13:22:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff02c04e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff02419f4b0 con 0x7ff024101800 2024-03-23T13:22:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff02c04e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff02419f670 con 0x7ff0241021c0 2024-03-23T13:22:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff029dea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff024100610 0x7ff0240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff029dea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff024100610 0x7ff0240693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34272/0 (socket says 172.21.15.7:34272) 2024-03-23T13:22:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff029dea640 1 -- 172.21.15.7:0/1106640639 learned_addr learned my addr 172.21.15.7:0/1106640639 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff0295e9640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff024101800 0x7ff02419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff02a5eb640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0241021c0 0x7ff02419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff029dea640 1 -- 172.21.15.7:0/1106640639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0241021c0 msgr2=0x7ff02419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff029dea640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0241021c0 0x7ff02419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff029dea640 1 -- 172.21.15.7:0/1106640639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff024101800 msgr2=0x7ff02419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff029dea640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff024101800 0x7ff02419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff029dea640 1 -- 172.21.15.7:0/1106640639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff020009560 con 0x7ff024100610 2024-03-23T13:22:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff02a5eb640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0241021c0 0x7ff02419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:22:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff0295e9640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff024101800 0x7ff02419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T13:22:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.318+0000 7ff029dea640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff024100610 0x7ff0240693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff020005ab0 tx=0x7ff020004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.319+0000 7ff01affd640 1 -- 172.21.15.7:0/1106640639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020004aa0 con 0x7ff024100610 2024-03-23T13:22:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.319+0000 7ff01affd640 1 -- 172.21.15.7:0/1106640639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0200052d0 con 0x7ff024100610 2024-03-23T13:22:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.319+0000 7ff02c04e640 1 -- 172.21.15.7:0/1106640639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff024196310 con 0x7ff024100610 2024-03-23T13:22:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.319+0000 7ff01affd640 1 -- 172.21.15.7:0/1106640639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020003e80 con 0x7ff024100610 2024-03-23T13:22:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.319+0000 7ff02c04e640 1 -- 172.21.15.7:0/1106640639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff024196820 con 0x7ff024100610 2024-03-23T13:22:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.320+0000 7ff02c04e640 1 -- 172.21.15.7:0/1106640639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff024107a50 con 0x7ff024100610 2024-03-23T13:22:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.321+0000 7ff01affd640 1 -- 172.21.15.7:0/1106640639 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7ff020005440 con 0x7ff024100610 2024-03-23T13:22:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.321+0000 7ff01affd640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff0100a20c0 0x7ff0100a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.322+0000 7ff0295e9640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff0100a20c0 0x7ff0100a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.322+0000 7ff01affd640 1 -- 172.21.15.7:0/1106640639 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8641+0+0 (secure 0 0 0) 0x7ff0200f0290 con 0x7ff024100610 2024-03-23T13:22:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.322+0000 7ff0295e9640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff0100a20c0 0x7ff0100a4580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff024069150 tx=0x7ff01400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:09.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.326+0000 7ff01affd640 1 -- 172.21.15.7:0/1106640639 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff0200b90c0 con 0x7ff024100610 2024-03-23T13:22:09.488 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g[73410]: ignoring --setuser ceph since I am not root 2024-03-23T13:22:09.489 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g[73410]: ignoring --setgroup ceph since I am not root 2024-03-23T13:22:09.489 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mds[73414]: -- 172.21.15.7:0/1624453188 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff63fc61a0 con 0x55ff63f9d000 2024-03-23T13:22:09.489 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g[73410]: starting mds.g at 2024-03-23T13:22:09.489 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g[73410]: 2024-03-23T13:22:09.234+0000 7fd158a27700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6838/0: (98) Address already in use 2024-03-23T13:22:09.489 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a[73759]: ignoring --setuser ceph since I am not root 2024-03-23T13:22:09.489 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a[73759]: ignoring --setgroup ceph since I am not root 2024-03-23T13:22:09.490 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mds[73763]: -- 172.21.15.7:0/3659319920 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560254f1a1a0 con 0x560254ef1000 2024-03-23T13:22:09.490 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a[73759]: starting mds.a at 2024-03-23T13:22:09.490 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a[32180]: 2024-03-23T13:22:09.187+0000 7fc00d7a5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T13:22:09.490 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mon[32184]: pgmap v420: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-03-23T13:22:09.490 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/4108718222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:22:09.490 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/2986857227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:22:09.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1192307549' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T13:22:09.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mon[32184]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T13:22:09.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mon[32184]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T13:22:09.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1192307549' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T13:22:09.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mon[32184]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T13:22:09.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:09 smithi007 ceph-mon[32184]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-23T13:22:09.541 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l[66225]: ignoring --setuser ceph since I am not root 2024-03-23T13:22:09.541 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l[66225]: ignoring --setgroup ceph since I am not root 2024-03-23T13:22:09.541 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mds[66229]: -- 172.21.15.155:0/750828735 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555baddde1a0 con 0x555baddb4800 2024-03-23T13:22:09.541 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l[66225]: starting mds.l at 2024-03-23T13:22:09.541 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l[66225]: 2024-03-23T13:22:09.246+0000 7f6a25696700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-23T13:22:09.542 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i[67167]: ignoring --setuser ceph since I am not root 2024-03-23T13:22:09.542 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i[67167]: ignoring --setgroup ceph since I am not root 2024-03-23T13:22:09.542 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mds[67171]: -- 172.21.15.155:0/2853316093 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ecd8121a0 con 0x560ecd7e9000 2024-03-23T13:22:09.542 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i[67167]: starting mds.i at 2024-03-23T13:22:09.542 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c[66850]: ignoring --setuser ceph since I am not root 2024-03-23T13:22:09.543 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c[66850]: ignoring --setgroup ceph since I am not root 2024-03-23T13:22:09.543 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mds[66854]: -- 172.21.15.155:0/2266624234 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5640ac9bf080 con 0x5640ac988800 2024-03-23T13:22:09.543 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mds[66854]: -- 172.21.15.155:0/2266624234 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640ac9b21a0 con 0x5640ac988800 2024-03-23T13:22:09.543 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c[66850]: starting mds.c at 2024-03-23T13:22:09.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mon[36157]: pgmap v420: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-03-23T13:22:09.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/4108718222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:22:09.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/2986857227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:22:09.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1192307549' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T13:22:09.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mon[36157]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T13:22:09.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mon[36157]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T13:22:09.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1192307549' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T13:22:09.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mon[36157]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T13:22:09.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mon[36157]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-23T13:22:09.544 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f[66540]: ignoring --setuser ceph since I am not root 2024-03-23T13:22:09.544 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f[66540]: ignoring --setgroup ceph since I am not root 2024-03-23T13:22:09.545 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mds[66544]: -- 172.21.15.155:0/1569460229 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d889939080 con 0x55d889902c00 2024-03-23T13:22:09.545 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-mds[66544]: -- 172.21.15.155:0/1569460229 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d88992c1a0 con 0x55d889902c00 2024-03-23T13:22:09.545 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:22:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f[66540]: starting mds.f at 2024-03-23T13:22:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:09.550+0000 7ff02c04e640 1 -- 172.21.15.7:0/1106640639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff02402d1d0 con 0x7ff024100610 2024-03-23T13:22:09.697 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h[66714]: ignoring --setuser ceph since I am not root 2024-03-23T13:22:09.698 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h[66714]: ignoring --setgroup ceph since I am not root 2024-03-23T13:22:09.698 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mds[66718]: -- 172.21.15.130:0/519562408 <== mon.2 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b68e9e9080 con 0x55b68e9b2c00 2024-03-23T13:22:09.698 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mds[66718]: -- 172.21.15.130:0/519562408 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b68e9dc1a0 con 0x55b68e9b2c00 2024-03-23T13:22:09.698 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h[66714]: starting mds.h at 2024-03-23T13:22:09.698 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k[66083]: ignoring --setuser ceph since I am not root 2024-03-23T13:22:09.699 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k[66083]: ignoring --setgroup ceph since I am not root 2024-03-23T13:22:09.699 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mds[66087]: -- 172.21.15.130:0/1845582052 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5652bf81e1a0 con 0x5652bf7f4c00 2024-03-23T13:22:09.699 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k[66083]: starting mds.k at 2024-03-23T13:22:09.699 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: ignoring --setuser ceph since I am not root 2024-03-23T13:22:09.699 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: ignoring --setgroup ceph since I am not root 2024-03-23T13:22:09.699 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mds[67037]: -- 172.21.15.130:0/2779734438 <== mon.2 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a56e7a1a0 con 0x560a56e51000 2024-03-23T13:22:09.700 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: starting mds.b at 2024-03-23T13:22:09.700 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e[66405]: ignoring --setuser ceph since I am not root 2024-03-23T13:22:09.700 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e[66405]: ignoring --setgroup ceph since I am not root 2024-03-23T13:22:09.700 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mds[66409]: -- 172.21.15.130:0/1794428559 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e2c2b41a0 con 0x564e2c28a800 2024-03-23T13:22:09.700 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e[66405]: starting mds.e at 2024-03-23T13:22:09.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mon[36225]: pgmap v420: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-03-23T13:22:09.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/4108718222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T13:22:09.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/2986857227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T13:22:09.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1192307549' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T13:22:09.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mon[36225]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T13:22:09.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mon[36225]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T13:22:09.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1192307549' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T13:22:09.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mon[36225]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T13:22:09.702 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:09 smithi130 ceph-mon[36225]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-23T13:22:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.189+0000 7ff01affd640 1 -- 172.21.15.7:0/1106640639 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7ff0200bc780 con 0x7ff024100610 2024-03-23T13:22:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 -- 172.21.15.7:0/1106640639 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff0100a20c0 msgr2=0x7ff0100a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff0100a20c0 0x7ff0100a4580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff024069150 tx=0x7ff01400b390 comp rx=0 tx=0).stop 2024-03-23T13:22:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 -- 172.21.15.7:0/1106640639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff024100610 msgr2=0x7ff0240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff024100610 0x7ff0240693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff020005ab0 tx=0x7ff020004270 comp rx=0 tx=0).stop 2024-03-23T13:22:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 -- 172.21.15.7:0/1106640639 shutdown_connections 2024-03-23T13:22:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7ff0100a20c0 0x7ff0100a4580 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0241021c0 0x7ff02419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff024101800 0x7ff02419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 --2- 172.21.15.7:0/1106640639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff024100610 0x7ff0240693e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 -- 172.21.15.7:0/1106640639 >> 172.21.15.7:0/1106640639 conn(0x7ff0240fbba0 msgr2=0x7ff024104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 -- 172.21.15.7:0/1106640639 shutdown_connections 2024-03-23T13:22:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.192+0000 7ff02c04e640 1 -- 172.21.15.7:0/1106640639 wait complete. 2024-03-23T13:22:10.210 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T13:22:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.318+0000 7f331ce74640 1 -- 172.21.15.7:0/1246706137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3318101800 msgr2=0x7f3318101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.318+0000 7f331ce74640 1 --2- 172.21.15.7:0/1246706137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3318101800 0x7f3318101c80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f330c009900 tx=0x7f330c02bc30 comp rx=0 tx=0).stop 2024-03-23T13:22:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 -- 172.21.15.7:0/1246706137 shutdown_connections 2024-03-23T13:22:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 --2- 172.21.15.7:0/1246706137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33181021c0 0x7f3318108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 --2- 172.21.15.7:0/1246706137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3318101800 0x7f3318101c80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 --2- 172.21.15.7:0/1246706137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3318100610 0x7f3318100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 -- 172.21.15.7:0/1246706137 >> 172.21.15.7:0/1246706137 conn(0x7f33180fbb80 msgr2=0x7f33180fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 -- 172.21.15.7:0/1246706137 shutdown_connections 2024-03-23T13:22:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 -- 172.21.15.7:0/1246706137 wait complete. 2024-03-23T13:22:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 Processor -- start 2024-03-23T13:22:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 -- start start 2024-03-23T13:22:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3318100610 0x7f3318069400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3318101800 0x7f331819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33181021c0 0x7f331819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f331819f300 con 0x7f3318101800 2024-03-23T13:22:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f331819f470 con 0x7f3318100610 2024-03-23T13:22:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.319+0000 7f331ce74640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f331819f630 con 0x7f33181021c0 2024-03-23T13:22:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3315d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3318101800 0x7f331819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3315d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3318101800 0x7f331819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34300/0 (socket says 172.21.15.7:34300) 2024-03-23T13:22:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3315d9b640 1 -- 172.21.15.7:0/3887914312 learned_addr learned my addr 172.21.15.7:0/3887914312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f331659c640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3318100610 0x7f3318069400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3316d9d640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33181021c0 0x7f331819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3315d9b640 1 -- 172.21.15.7:0/3887914312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33181021c0 msgr2=0x7f331819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3315d9b640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33181021c0 0x7f331819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3315d9b640 1 -- 172.21.15.7:0/3887914312 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3318100610 msgr2=0x7f3318069400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3315d9b640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3318100610 0x7f3318069400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3315d9b640 1 -- 172.21.15.7:0/3887914312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c009560 con 0x7f3318101800 2024-03-23T13:22:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3316d9d640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33181021c0 0x7f331819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T13:22:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f3315d9b640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3318101800 0x7f331819c530 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f330c02c6a0 tx=0x7f330c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f33077fe640 1 -- 172.21.15.7:0/3887914312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c0044e0 con 0x7f3318101800 2024-03-23T13:22:10.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f33077fe640 1 -- 172.21.15.7:0/3887914312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f330c0054e0 con 0x7f3318101800 2024-03-23T13:22:10.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f331ce74640 1 -- 172.21.15.7:0/3887914312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331819f8b0 con 0x7f3318101800 2024-03-23T13:22:10.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f33077fe640 1 -- 172.21.15.7:0/3887914312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c03e450 con 0x7f3318101800 2024-03-23T13:22:10.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.320+0000 7f331ce74640 1 -- 172.21.15.7:0/3887914312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331819fdc0 con 0x7f3318101800 2024-03-23T13:22:10.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.321+0000 7f331ce74640 1 -- 172.21.15.7:0/3887914312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3318107930 con 0x7f3318101800 2024-03-23T13:22:10.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.323+0000 7f33077fe640 1 -- 172.21.15.7:0/3887914312 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f330c003710 con 0x7f3318101800 2024-03-23T13:22:10.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.323+0000 7f33077fe640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f33000a20c0 0x7f33000a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:10.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.324+0000 7f331659c640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f33000a20c0 0x7f33000a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:10.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.324+0000 7f33077fe640 1 -- 172.21.15.7:0/3887914312 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8641+0+0 (secure 0 0 0) 0x7f330c0f0cb0 con 0x7f3318101800 2024-03-23T13:22:10.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.324+0000 7f331659c640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f33000a20c0 0x7f33000a4580 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3318101660 tx=0x7f331000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.328+0000 7f33077fe640 1 -- 172.21.15.7:0/3887914312 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f330c0b9c00 con 0x7f3318101800 2024-03-23T13:22:10.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1106640639' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T13:22:10.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T13:22:10.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T13:22:10.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: Cluster is now healthy 2024-03-23T13:22:10.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/1106640639' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T13:22:10.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.130:6832/660436653,v1:172.21.15.130:6834/660436653] up:boot 2024-03-23T13:22:10.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.7:6838/1406800581,v1:172.21.15.7:6840/1406800581] up:boot 2024-03-23T13:22:10.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.7:6839/4146821800,v1:172.21.15.7:6841/4146821800] up:boot 2024-03-23T13:22:10.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.130:6837/3709073683,v1:172.21.15.130:6839/3709073683] up:boot 2024-03-23T13:22:10.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6835/3407613907,v1:172.21.15.155:6839/3407613907] up:boot 2024-03-23T13:22:10.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.130:6833/540070869,v1:172.21.15.130:6835/540070869] up:boot 2024-03-23T13:22:10.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6832/1490733946,v1:172.21.15.155:6836/1490733946] up:boot 2024-03-23T13:22:10.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.130:6836/1974054382,v1:172.21.15.130:6838/1974054382] up:boot 2024-03-23T13:22:10.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6834/3191807399,v1:172.21.15.155:6838/3191807399] up:boot 2024-03-23T13:22:10.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: mds.? [v2:172.21.15.155:6833/676418363,v1:172.21.15.155:6837/676418363] up:boot 2024-03-23T13:22:10.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: fsmap 12 up:standby 2024-03-23T13:22:10.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T13:22:10.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T13:22:10.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T13:22:10.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T13:22:10.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T13:22:10.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T13:22:10.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T13:22:10.468 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T13:22:10.468 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T13:22:10.468 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:10 smithi007 ceph-mon[32184]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T13:22:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:10.514+0000 7f331ce74640 1 -- 172.21.15.7:0/3887914312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f331810c070 con 0x7f3318101800 2024-03-23T13:22:10.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1106640639' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T13:22:10.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T13:22:10.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T13:22:10.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: Cluster is now healthy 2024-03-23T13:22:10.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/1106640639' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T13:22:10.699 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.130:6832/660436653,v1:172.21.15.130:6834/660436653] up:boot 2024-03-23T13:22:10.699 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.7:6838/1406800581,v1:172.21.15.7:6840/1406800581] up:boot 2024-03-23T13:22:10.699 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.7:6839/4146821800,v1:172.21.15.7:6841/4146821800] up:boot 2024-03-23T13:22:10.699 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.130:6837/3709073683,v1:172.21.15.130:6839/3709073683] up:boot 2024-03-23T13:22:10.699 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6835/3407613907,v1:172.21.15.155:6839/3407613907] up:boot 2024-03-23T13:22:10.699 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.130:6833/540070869,v1:172.21.15.130:6835/540070869] up:boot 2024-03-23T13:22:10.699 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6832/1490733946,v1:172.21.15.155:6836/1490733946] up:boot 2024-03-23T13:22:10.699 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.130:6836/1974054382,v1:172.21.15.130:6838/1974054382] up:boot 2024-03-23T13:22:10.700 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6834/3191807399,v1:172.21.15.155:6838/3191807399] up:boot 2024-03-23T13:22:10.700 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: mds.? [v2:172.21.15.155:6833/676418363,v1:172.21.15.155:6837/676418363] up:boot 2024-03-23T13:22:10.700 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: fsmap 12 up:standby 2024-03-23T13:22:10.700 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T13:22:10.700 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T13:22:10.700 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T13:22:10.700 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T13:22:10.700 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T13:22:10.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T13:22:10.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T13:22:10.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T13:22:10.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T13:22:10.701 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:10 smithi130 ceph-mon[36225]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T13:22:10.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1106640639' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T13:22:10.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T13:22:10.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T13:22:10.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: Cluster is now healthy 2024-03-23T13:22:10.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/1106640639' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T13:22:10.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.130:6832/660436653,v1:172.21.15.130:6834/660436653] up:boot 2024-03-23T13:22:10.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.7:6838/1406800581,v1:172.21.15.7:6840/1406800581] up:boot 2024-03-23T13:22:10.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.7:6839/4146821800,v1:172.21.15.7:6841/4146821800] up:boot 2024-03-23T13:22:10.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.130:6837/3709073683,v1:172.21.15.130:6839/3709073683] up:boot 2024-03-23T13:22:10.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6835/3407613907,v1:172.21.15.155:6839/3407613907] up:boot 2024-03-23T13:22:10.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.130:6833/540070869,v1:172.21.15.130:6835/540070869] up:boot 2024-03-23T13:22:10.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6832/1490733946,v1:172.21.15.155:6836/1490733946] up:boot 2024-03-23T13:22:10.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.130:6836/1974054382,v1:172.21.15.130:6838/1974054382] up:boot 2024-03-23T13:22:10.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6834/3191807399,v1:172.21.15.155:6838/3191807399] up:boot 2024-03-23T13:22:10.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: mds.? [v2:172.21.15.155:6833/676418363,v1:172.21.15.155:6837/676418363] up:boot 2024-03-23T13:22:10.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: fsmap 12 up:standby 2024-03-23T13:22:10.794 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T13:22:10.794 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T13:22:10.794 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T13:22:10.794 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T13:22:10.794 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T13:22:10.794 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T13:22:10.794 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T13:22:10.795 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T13:22:10.795 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T13:22:10.795 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:10 smithi155 ceph-mon[36157]: from='mgr.14152 172.21.15.7:0/2532482165' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T13:22:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.365+0000 7f33077fe640 1 -- 172.21.15.7:0/3887914312 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7f330c043070 con 0x7f3318101800 2024-03-23T13:22:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-03-23T13:22:11.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.368+0000 7f331ce74640 1 -- 172.21.15.7:0/3887914312 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f33000a20c0 msgr2=0x7f33000a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.368+0000 7f331ce74640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f33000a20c0 0x7f33000a4580 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3318101660 tx=0x7f331000b390 comp rx=0 tx=0).stop 2024-03-23T13:22:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.368+0000 7f331ce74640 1 -- 172.21.15.7:0/3887914312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3318101800 msgr2=0x7f331819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.368+0000 7f331ce74640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3318101800 0x7f331819c530 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f330c02c6a0 tx=0x7f330c0042f0 comp rx=0 tx=0).stop 2024-03-23T13:22:11.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.369+0000 7f331ce74640 1 -- 172.21.15.7:0/3887914312 shutdown_connections 2024-03-23T13:22:11.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.369+0000 7f331ce74640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f33000a20c0 0x7f33000a4580 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:11.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.369+0000 7f331ce74640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33181021c0 0x7f331819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:11.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.369+0000 7f331ce74640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3318101800 0x7f331819c530 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:11.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.369+0000 7f331ce74640 1 --2- 172.21.15.7:0/3887914312 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3318100610 0x7f3318069400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:11.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.369+0000 7f331ce74640 1 -- 172.21.15.7:0/3887914312 >> 172.21.15.7:0/3887914312 conn(0x7f33180fbb80 msgr2=0x7f3318104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:11.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.369+0000 7f331ce74640 1 -- 172.21.15.7:0/3887914312 shutdown_connections 2024-03-23T13:22:11.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.369+0000 7f331ce74640 1 -- 172.21.15.7:0/3887914312 wait complete. 2024-03-23T13:22:11.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T13:22:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.498+0000 7f53ff809640 1 -- 172.21.15.7:0/1079832719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f8101810 msgr2=0x7f53f8101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.498+0000 7f53ff809640 1 --2- 172.21.15.7:0/1079832719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f8101810 0x7f53f8101c90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009900 tx=0x7f53ec02bc30 comp rx=0 tx=0).stop 2024-03-23T13:22:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.498+0000 7f53ff809640 1 -- 172.21.15.7:0/1079832719 shutdown_connections 2024-03-23T13:22:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.498+0000 7f53ff809640 1 --2- 172.21.15.7:0/1079832719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f81021d0 0x7f53f8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.498+0000 7f53ff809640 1 --2- 172.21.15.7:0/1079832719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f8101810 0x7f53f8101c90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.498+0000 7f53ff809640 1 --2- 172.21.15.7:0/1079832719 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53f8100620 0x7f53f8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.498+0000 7f53ff809640 1 -- 172.21.15.7:0/1079832719 >> 172.21.15.7:0/1079832719 conn(0x7f53f80fbbb0 msgr2=0x7f53f80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53ff809640 1 -- 172.21.15.7:0/1079832719 shutdown_connections 2024-03-23T13:22:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53ff809640 1 -- 172.21.15.7:0/1079832719 wait complete. 2024-03-23T13:22:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53ff809640 1 Processor -- start 2024-03-23T13:22:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53ff809640 1 -- start start 2024-03-23T13:22:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53ff809640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f8100620 0x7f53f81977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53ff809640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53f8101810 0x7f53f8197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53ff809640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f81021d0 0x7f53f819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53ff809640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53f819f2b0 con 0x7f53f8100620 2024-03-23T13:22:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53ff809640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f53f819f420 con 0x7f53f8101810 2024-03-23T13:22:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53ff809640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53f819f5e0 con 0x7f53f81021d0 2024-03-23T13:22:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53fd5a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f8100620 0x7f53f81977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53fd5a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f8100620 0x7f53f81977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34306/0 (socket says 172.21.15.7:34306) 2024-03-23T13:22:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.499+0000 7f53fd5a5640 1 -- 172.21.15.7:0/3668893366 learned_addr learned my addr 172.21.15.7:0/3668893366 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-23T13:22:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53fcda4640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53f8101810 0x7f53f8197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53fd5a5640 1 -- 172.21.15.7:0/3668893366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f81021d0 msgr2=0x7f53f819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53fd5a5640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f81021d0 0x7f53f819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53fd5a5640 1 -- 172.21.15.7:0/3668893366 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53f8101810 msgr2=0x7f53f8197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53fd5a5640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53f8101810 0x7f53f8197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53fd5a5640 1 -- 172.21.15.7:0/3668893366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec009560 con 0x7f53f8100620 2024-03-23T13:22:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53fcda4640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53f8101810 0x7f53f8197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T13:22:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53fd5a5640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f8100620 0x7f53f81977c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f53f000d950 tx=0x7f53f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53f67fc640 1 -- 172.21.15.7:0/3668893366 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0009c30 con 0x7f53f8100620 2024-03-23T13:22:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53f67fc640 1 -- 172.21.15.7:0/3668893366 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f0009d90 con 0x7f53f8100620 2024-03-23T13:22:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53ff809640 1 -- 172.21.15.7:0/3668893366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f819f8c0 con 0x7f53f8100620 2024-03-23T13:22:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53f67fc640 1 -- 172.21.15.7:0/3668893366 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f000f880 con 0x7f53f8100620 2024-03-23T13:22:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.500+0000 7f53ff809640 1 -- 172.21.15.7:0/3668893366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f819fdd0 con 0x7f53f8100620 2024-03-23T13:22:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.501+0000 7f53ff809640 1 -- 172.21.15.7:0/3668893366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f8107940 con 0x7f53f8100620 2024-03-23T13:22:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.502+0000 7f53f67fc640 1 -- 172.21.15.7:0/3668893366 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 136755+0+0 (secure 0 0 0) 0x7f53f000f9e0 con 0x7f53f8100620 2024-03-23T13:22:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.502+0000 7f53f67fc640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f53e00a20c0 0x7f53e00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T13:22:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.503+0000 7f53fcda4640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f53e00a20c0 0x7f53e00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T13:22:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.503+0000 7f53f67fc640 1 -- 172.21.15.7:0/3668893366 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8116+0+0 (secure 0 0 0) 0x7f53f00cdf90 con 0x7f53f8100620 2024-03-23T13:22:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.503+0000 7f53fcda4640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f53e00a20c0 0x7f53e00a4580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f53ec036040 tx=0x7f53ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T13:22:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.507+0000 7f53f67fc640 1 -- 172.21.15.7:0/3668893366 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f53f0096f80 con 0x7f53f8100620 2024-03-23T13:22:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:11.690+0000 7f53ff809640 1 -- 172.21.15.7:0/3668893366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f53f810d130 con 0x7f53f8100620 2024-03-23T13:22:11.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:11 smithi130 ceph-mon[36225]: pgmap v422: 97 pgs: 97 active+clean; 47 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-23T13:22:11.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:11 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3887914312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T13:22:11.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:11 smithi155 ceph-mon[36157]: pgmap v422: 97 pgs: 97 active+clean; 47 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-23T13:22:11.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:11 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3887914312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T13:22:11.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:11 smithi007 ceph-mon[32184]: pgmap v422: 97 pgs: 97 active+clean; 47 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-23T13:22:11.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:11 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3887914312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T13:22:12.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.370+0000 7f53f67fc640 1 -- 172.21.15.7:0/3668893366 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7f53f009a640 con 0x7f53f8100620 2024-03-23T13:22:12.373 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-03-23T13:22:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.374+0000 7f53ff809640 1 -- 172.21.15.7:0/3668893366 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f53e00a20c0 msgr2=0x7f53e00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.374+0000 7f53ff809640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f53e00a20c0 0x7f53e00a4580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f53ec036040 tx=0x7f53ec000bc0 comp rx=0 tx=0).stop 2024-03-23T13:22:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.374+0000 7f53ff809640 1 -- 172.21.15.7:0/3668893366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f8100620 msgr2=0x7f53f81977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T13:22:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.374+0000 7f53ff809640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f8100620 0x7f53f81977c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f53f000d950 tx=0x7f53f000de10 comp rx=0 tx=0).stop 2024-03-23T13:22:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.374+0000 7f53ff809640 1 -- 172.21.15.7:0/3668893366 shutdown_connections 2024-03-23T13:22:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.374+0000 7f53ff809640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.7:6800/1535208045,v1:172.21.15.7:6801/1535208045] conn(0x7f53e00a20c0 0x7f53e00a4580 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.374+0000 7f53ff809640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f81021d0 0x7f53f819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.375+0000 7f53ff809640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53f8101810 0x7f53f8197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T13:22:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.375+0000 7f53ff809640 1 --2- 172.21.15.7:0/3668893366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f8100620 0x7f53f81977c0 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f53f000d950 tx=0x7f53f000de10 comp rx=0 tx=0).stop 2024-03-23T13:22:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.375+0000 7f53ff809640 1 -- 172.21.15.7:0/3668893366 >> 172.21.15.7:0/3668893366 conn(0x7f53f80fbbb0 msgr2=0x7f53f8104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T13:22:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.375+0000 7f53ff809640 1 -- 172.21.15.7:0/3668893366 shutdown_connections 2024-03-23T13:22:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-23T13:22:12.375+0000 7f53ff809640 1 -- 172.21.15.7:0/3668893366 wait complete. 2024-03-23T13:22:12.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T13:22:12.431 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T13:22:12.464 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T13:22:12.498 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-23T13:22:12.498 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T13:22:12.513 DEBUG:teuthology.orchestra.run.smithi130:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T13:22:12.530 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T13:22:12.553 INFO:tasks.cephadm:Stopping all daemons... 2024-03-23T13:22:12.554 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-23T13:22:12.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.a 2024-03-23T13:22:12.697 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:12 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3887914312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T13:22:12.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:12 smithi130 ceph-mon[36225]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T13:22:12.698 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:12 smithi130 ceph-mon[36225]: from='client.? 172.21.15.7:0/3668893366' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T13:22:12.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:12 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3887914312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T13:22:12.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:12 smithi155 ceph-mon[36157]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T13:22:12.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:12 smithi155 ceph-mon[36157]: from='client.? 172.21.15.7:0/3668893366' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T13:22:12.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:12 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3887914312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T13:22:12.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:12 smithi007 ceph-mon[32184]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T13:22:12.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:12 smithi007 ceph-mon[32184]: from='client.? 172.21.15.7:0/3668893366' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T13:22:12.831 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:12 smithi007 systemd[1]: Stopping Ceph mon.a for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a[32180]: 2024-03-23T13:22:12.860+0000 7fc012fb0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:22:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a[32180]: 2024-03-23T13:22:12.860+0000 7fc012fb0700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-23T13:22:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:12 smithi007 podman[93357]: 2024-03-23 13:22:12.881683192 +0000 UTC m=+0.134880194 container died d3f413d67b18980d577dbd7fb25ee5dacd8eb8b701a81b1ddfb2c69a27b4bc53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319) 2024-03-23T13:22:13.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:13 smithi007 podman[93357]: 2024-03-23 13:22:13.377022305 +0000 UTC m=+0.630219347 container cleanup d3f413d67b18980d577dbd7fb25ee5dacd8eb8b701a81b1ddfb2c69a27b4bc53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:22:13.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 23 13:22:13 smithi007 bash[93357]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-a 2024-03-23T13:22:13.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.a.service' 2024-03-23T13:22:13.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:22:13.862 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-23T13:22:13.863 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-23T13:22:13.863 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.b 2024-03-23T13:22:14.197 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:13 smithi130 systemd[1]: Stopping Ceph mon.b for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:14.523 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:14 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-b[36221]: 2024-03-23T13:22:14.254+0000 7fd98b68a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:22:14.523 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:14 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-b[36221]: 2024-03-23T13:22:14.254+0000 7fd98b68a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-23T13:22:14.850 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:14 smithi130 podman[70014]: 2024-03-23 13:22:14.520955906 +0000 UTC m=+0.474607102 container died 42a461ed95f7fefd5b0ab3ac10bffeada1e34618f438e3ba634846b6e79cd078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-b, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-23T13:22:15.197 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:15 smithi130 podman[70014]: 2024-03-23 13:22:15.023040021 +0000 UTC m=+0.976691253 container cleanup 42a461ed95f7fefd5b0ab3ac10bffeada1e34618f438e3ba634846b6e79cd078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-b, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-23T13:22:15.198 INFO:journalctl@ceph.mon.b.smithi130.stdout:Mar 23 13:22:15 smithi130 bash[70014]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-b 2024-03-23T13:22:15.479 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.b.service' 2024-03-23T13:22:15.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:22:15.532 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-23T13:22:15.532 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-23T13:22:15.532 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.c 2024-03-23T13:22:15.848 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:15 smithi155 systemd[1]: Stopping Ceph mon.c for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:15 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-c[36153]: 2024-03-23T13:22:15.844+0000 7f446c4d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:22:16.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:15 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-c[36153]: 2024-03-23T13:22:15.844+0000 7f446c4d4700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-23T13:22:16.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:15 smithi155 podman[70171]: 2024-03-23 13:22:15.957422791 +0000 UTC m=+0.239726890 container died 7ab6cf0d7485c22cb5d9eaa87dd1bce4ec1b09a06c2f1ab9470ee7203bf92852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:22:16.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:16 smithi155 podman[70171]: 2024-03-23 13:22:16.504404457 +0000 UTC m=+0.786708552 container cleanup 7ab6cf0d7485c22cb5d9eaa87dd1bce4ec1b09a06c2f1ab9470ee7203bf92852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD) 2024-03-23T13:22:16.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 23 13:22:16 smithi155 bash[70171]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mon-c 2024-03-23T13:22:17.005 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mon.c.service' 2024-03-23T13:22:17.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:22:17.055 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-23T13:22:17.055 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-23T13:22:17.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.x 2024-03-23T13:22:17.359 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:22:17 smithi007 systemd[1]: Stopping Ceph mgr.x for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:17.693 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:22:17 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x[32416]: 2024-03-23T13:22:17.356+0000 7fb62d79c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:22:17.693 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:22:17 smithi007 podman[93478]: 2024-03-23 13:22:17.387493114 +0000 UTC m=+0.145163909 container died 67d150695aee77b534d96595087965c6e69f62c5976b69a1dddac7a3c4a1d72f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-23T13:22:18.028 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:22:17 smithi007 podman[93478]: 2024-03-23 13:22:17.790628413 +0000 UTC m=+0.548299205 container cleanup 67d150695aee77b534d96595087965c6e69f62c5976b69a1dddac7a3c4a1d72f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:22:18.028 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 23 13:22:17 smithi007 bash[93478]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-x 2024-03-23T13:22:18.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-23T13:22:18.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:22:18.193 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-23T13:22:18.194 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-23T13:22:18.194 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.y 2024-03-23T13:22:18.502 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:22:18 smithi130 systemd[1]: Stopping Ceph mgr.y for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:18.823 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:22:18 smithi130 podman[70137]: 2024-03-23 13:22:18.500250053 +0000 UTC m=+0.122762974 container died 0b3e73965526f9db88c16c2d21a6ab89f525a45a1926e05acc8727eea2c0c5ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:22:19.159 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:22:18 smithi130 podman[70137]: 2024-03-23 13:22:18.920840936 +0000 UTC m=+0.543353842 container cleanup 0b3e73965526f9db88c16c2d21a6ab89f525a45a1926e05acc8727eea2c0c5ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-03-23T13:22:19.159 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:22:18 smithi130 bash[70137]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y 2024-03-23T13:22:19.386 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-23T13:22:19.410 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:22:19 smithi130 podman[70151]: 2024-03-23 13:22:19.15596731 +0000 UTC m=+0.650163057 container remove 0b3e73965526f9db88c16c2d21a6ab89f525a45a1926e05acc8727eea2c0c5ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5) 2024-03-23T13:22:19.410 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:22:19 smithi130 systemd[1]: ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-23T13:22:19.410 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:22:19 smithi130 systemd[1]: ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.y.service: Failed with result 'exit-code'. 2024-03-23T13:22:19.411 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:22:19 smithi130 systemd[1]: Stopped Ceph mgr.y for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:22:19.411 INFO:journalctl@ceph.mgr.y.smithi130.stdout:Mar 23 13:22:19 smithi130 systemd[1]: ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.y.service: Consumed 10.161s CPU time. 2024-03-23T13:22:19.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:22:19.423 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-23T13:22:19.424 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-23T13:22:19.424 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.z 2024-03-23T13:22:19.766 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:22:19 smithi155 systemd[1]: Stopping Ceph mgr.z for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:20.041 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:22:19 smithi155 podman[70293]: 2024-03-23 13:22:19.763346518 +0000 UTC m=+0.152534321 container died 3499c355819435b9ff05ad5090cf11c468734b63cf57f560d95d4370b74c3157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:22:20.531 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:22:20 smithi155 podman[70293]: 2024-03-23 13:22:20.268881544 +0000 UTC m=+0.658069351 container cleanup 3499c355819435b9ff05ad5090cf11c468734b63cf57f560d95d4370b74c3157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-23T13:22:20.532 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 23 13:22:20 smithi155 bash[70293]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mgr-z 2024-03-23T13:22:20.712 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-23T13:22:20.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:22:20.752 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-23T13:22:20.752 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-23T13:22:20.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.0 2024-03-23T13:22:21.080 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:20 smithi007 systemd[1]: Stopping Ceph osd.0 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:21.580 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:21 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0[47251]: 2024-03-23T13:22:21.078+0000 7f8e73f36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:22:21.580 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:21 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0[47251]: 2024-03-23T13:22:21.078+0000 7f8e73f36700 -1 osd.0 90 *** Got signal Terminated *** 2024-03-23T13:22:21.580 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:21 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0[47251]: 2024-03-23T13:22:21.078+0000 7f8e73f36700 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:22:26.925 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:26 smithi007 podman[93599]: 2024-03-23 13:22:26.619355326 +0000 UTC m=+5.674923902 container died e61efa28d77b22c61f7a5639e0b1fed66521b492c5c063a4b969b6803e146022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-23T13:22:27.176 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:27 smithi007 podman[93599]: 2024-03-23 13:22:27.162378304 +0000 UTC m=+6.217946877 container cleanup e61efa28d77b22c61f7a5639e0b1fed66521b492c5c063a4b969b6803e146022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-23T13:22:27.176 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:27 smithi007 bash[93599]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0 2024-03-23T13:22:27.830 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:27 smithi007 podman[93619]: 2024-03-23 13:22:27.38054739 +0000 UTC m=+0.756888459 container remove e61efa28d77b22c61f7a5639e0b1fed66521b492c5c063a4b969b6803e146022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:22:28.120 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:27 smithi007 podman[93712]: 2024-03-23 13:22:27.734630605 +0000 UTC m=+0.018201747 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:22:28.120 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:27 smithi007 podman[93712]: 2024-03-23 13:22:27.852205912 +0000 UTC m=+0.135777024 container create f83b638f49cd054ec6710b06f998cee913d0c15b8f7f0c62e33a415ce0891edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-23T13:22:28.434 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:28 smithi007 podman[93712]: 2024-03-23 13:22:28.117589493 +0000 UTC m=+0.401160618 container init f83b638f49cd054ec6710b06f998cee913d0c15b8f7f0c62e33a415ce0891edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-23T13:22:28.434 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:28 smithi007 podman[93712]: 2024-03-23 13:22:28.120916432 +0000 UTC m=+0.404487549 container start f83b638f49cd054ec6710b06f998cee913d0c15b8f7f0c62e33a415ce0891edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-03-23T13:22:28.434 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:28 smithi007 podman[93712]: 2024-03-23 13:22:28.20950698 +0000 UTC m=+0.493078123 container attach f83b638f49cd054ec6710b06f998cee913d0c15b8f7f0c62e33a415ce0891edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-23T13:22:28.434 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 23 13:22:28 smithi007 podman[93712]: 2024-03-23 13:22:28.431489768 +0000 UTC m=+0.715060883 container died f83b638f49cd054ec6710b06f998cee913d0c15b8f7f0c62e33a415ce0891edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:22:29.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.0.service' 2024-03-23T13:22:29.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:22:29.098 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-23T13:22:29.098 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-23T13:22:29.098 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.1 2024-03-23T13:22:29.412 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:29 smithi130 systemd[1]: Stopping Ceph osd.1 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:29.698 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:29 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1[40822]: 2024-03-23T13:22:29.408+0000 7fc7ec2e9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:22:29.698 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:29 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1[40822]: 2024-03-23T13:22:29.408+0000 7fc7ec2e9700 -1 osd.1 90 *** Got signal Terminated *** 2024-03-23T13:22:29.698 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:29 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1[40822]: 2024-03-23T13:22:29.408+0000 7fc7ec2e9700 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:22:35.198 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:34 smithi130 podman[70262]: 2024-03-23 13:22:34.93681446 +0000 UTC m=+5.645115922 container died 2451974fed842161f732f4da7d18310c36c6fbd0a7dc8aa4db9212f7b71aee17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-23T13:22:35.618 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:35 smithi130 podman[70262]: 2024-03-23 13:22:35.434242653 +0000 UTC m=+6.142544115 container cleanup 2451974fed842161f732f4da7d18310c36c6fbd0a7dc8aa4db9212f7b71aee17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-23T13:22:35.618 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:35 smithi130 bash[70262]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1 2024-03-23T13:22:35.947 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:35 smithi130 podman[70280]: 2024-03-23 13:22:35.694357473 +0000 UTC m=+0.758771210 container remove 2451974fed842161f732f4da7d18310c36c6fbd0a7dc8aa4db9212f7b71aee17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-23T13:22:36.284 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:36 smithi130 podman[70380]: 2024-03-23 13:22:36.028701706 +0000 UTC m=+0.102997563 container create dcb790797effa2fc4afe3f8f7542d4a5d24cebf196fae3767ce740b509fc64ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2) 2024-03-23T13:22:36.284 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:36 smithi130 podman[70380]: 2024-03-23 13:22:35.943599874 +0000 UTC m=+0.017895744 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:22:36.590 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:36 smithi130 podman[70380]: 2024-03-23 13:22:36.281776627 +0000 UTC m=+0.356072499 container init dcb790797effa2fc4afe3f8f7542d4a5d24cebf196fae3767ce740b509fc64ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-03-23T13:22:36.590 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:36 smithi130 podman[70380]: 2024-03-23 13:22:36.2857291 +0000 UTC m=+0.360024957 container start dcb790797effa2fc4afe3f8f7542d4a5d24cebf196fae3767ce740b509fc64ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:22:36.590 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:36 smithi130 podman[70380]: 2024-03-23 13:22:36.373502552 +0000 UTC m=+0.447798426 container attach dcb790797effa2fc4afe3f8f7542d4a5d24cebf196fae3767ce740b509fc64ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-23T13:22:36.948 INFO:journalctl@ceph.osd.1.smithi130.stdout:Mar 23 13:22:36 smithi130 podman[70380]: 2024-03-23 13:22:36.587612086 +0000 UTC m=+0.661907954 container died dcb790797effa2fc4afe3f8f7542d4a5d24cebf196fae3767ce740b509fc64ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-23T13:22:37.260 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.1.service' 2024-03-23T13:22:37.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:22:37.305 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-23T13:22:37.305 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-23T13:22:37.305 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.2 2024-03-23T13:22:37.626 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:37 smithi155 systemd[1]: Stopping Ceph osd.2 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:38.041 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:37 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2[41320]: 2024-03-23T13:22:37.622+0000 7f8d2d1a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:22:38.041 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:37 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2[41320]: 2024-03-23T13:22:37.622+0000 7f8d2d1a0700 -1 osd.2 90 *** Got signal Terminated *** 2024-03-23T13:22:38.041 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:37 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2[41320]: 2024-03-23T13:22:37.622+0000 7f8d2d1a0700 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:22:43.477 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:43 smithi155 podman[70428]: 2024-03-23 13:22:43.163858231 +0000 UTC m=+5.666599908 container died 4e675c037796093f3dab6896e55e29764efa016df162b963eff27412cfc61757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:22:43.791 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:43 smithi155 podman[70428]: 2024-03-23 13:22:43.665528558 +0000 UTC m=+6.168270226 container cleanup 4e675c037796093f3dab6896e55e29764efa016df162b963eff27412cfc61757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-23T13:22:43.792 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:43 smithi155 bash[70428]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2 2024-03-23T13:22:44.277 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:43 smithi155 podman[70445]: 2024-03-23 13:22:43.950940913 +0000 UTC m=+0.778276258 container remove 4e675c037796093f3dab6896e55e29764efa016df162b963eff27412cfc61757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-23T13:22:44.541 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:44 smithi155 podman[70534]: 2024-03-23 13:22:44.1775321 +0000 UTC m=+0.018034077 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:22:44.541 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:44 smithi155 podman[70534]: 2024-03-23 13:22:44.301861215 +0000 UTC m=+0.142363189 container create 299f8eac703e9f03242c432e6b36ec0fd559485f7e2e3601a7b6ac746f05b0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:22:44.866 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:44 smithi155 podman[70534]: 2024-03-23 13:22:44.571913608 +0000 UTC m=+0.412415582 container init 299f8eac703e9f03242c432e6b36ec0fd559485f7e2e3601a7b6ac746f05b0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:22:44.866 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:44 smithi155 podman[70534]: 2024-03-23 13:22:44.574989317 +0000 UTC m=+0.415491286 container start 299f8eac703e9f03242c432e6b36ec0fd559485f7e2e3601a7b6ac746f05b0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:22:44.866 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:44 smithi155 podman[70534]: 2024-03-23 13:22:44.662939821 +0000 UTC m=+0.503441809 container attach 299f8eac703e9f03242c432e6b36ec0fd559485f7e2e3601a7b6ac746f05b0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:22:45.177 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 23 13:22:44 smithi155 podman[70534]: 2024-03-23 13:22:44.863971658 +0000 UTC m=+0.704473634 container died 299f8eac703e9f03242c432e6b36ec0fd559485f7e2e3601a7b6ac746f05b0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2) 2024-03-23T13:22:45.504 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.2.service' 2024-03-23T13:22:45.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:22:45.545 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-23T13:22:45.545 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-23T13:22:45.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.3 2024-03-23T13:22:45.830 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:45 smithi007 systemd[1]: Stopping Ceph osd.3 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:46.330 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:45 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3[53764]: 2024-03-23T13:22:45.864+0000 7f2be5bc8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:22:46.331 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:45 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3[53764]: 2024-03-23T13:22:45.864+0000 7f2be5bc8700 -1 osd.3 90 *** Got signal Terminated *** 2024-03-23T13:22:46.331 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:45 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3[53764]: 2024-03-23T13:22:45.864+0000 7f2be5bc8700 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:22:47.698 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:47.383+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:48.011 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:47 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:47.731+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:48.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:47 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:47.911+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:48.330 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:48 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3[53764]: 2024-03-23T13:22:48.007+0000 7f2bde1b4700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:22.855893+0000 front 2024-03-23T13:22:22.855981+0000 (oldest deadline 2024-03-23T13:22:47.556021+0000) 2024-03-23T13:22:48.698 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:48 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:48.341+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:48.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:48 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:48.402+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:49.080 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:48 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3[53764]: 2024-03-23T13:22:48.960+0000 7f2bde1b4700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:22.855893+0000 front 2024-03-23T13:22:22.855981+0000 (oldest deadline 2024-03-23T13:22:47.556021+0000) 2024-03-23T13:22:49.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:48 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:48.712+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:49.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:48 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:48.918+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:49.698 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:49 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:49.332+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:49.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:49 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:49.428+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:49.987 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:49 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:49.700+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:50.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:49 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:49.956+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:50.330 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:49 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3[53764]: 2024-03-23T13:22:49.984+0000 7f2bde1b4700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:22.855893+0000 front 2024-03-23T13:22:22.855981+0000 (oldest deadline 2024-03-23T13:22:47.556021+0000) 2024-03-23T13:22:50.698 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:50 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:50.376+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:50.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:50.392+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:50.830 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:50 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:50.652+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:50.830 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:50 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:50.441+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:22:51.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:50.937+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:51.666 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:51 smithi007 podman[93830]: 2024-03-23 13:22:51.38583128 +0000 UTC m=+5.639509545 container died ae287f814aa9f79c2aacc33cc3e20feda88583e9384b99ea044bd58c8959cc80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-23T13:22:51.666 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:51 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:51.408+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:22:51.698 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:51 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:51.337+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:51.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:51 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:51.416+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:52.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:51 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:51.663+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:52.081 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:51 smithi007 podman[93830]: 2024-03-23 13:22:51.823843065 +0000 UTC m=+6.077521365 container cleanup ae287f814aa9f79c2aacc33cc3e20feda88583e9384b99ea044bd58c8959cc80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-23T13:22:52.081 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:51 smithi007 bash[93830]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3 2024-03-23T13:22:52.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:51 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:51.970+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:52.382 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:52 smithi007 podman[93847]: 2024-03-23 13:22:52.083695713 +0000 UTC m=+0.698011912 container remove ae287f814aa9f79c2aacc33cc3e20feda88583e9384b99ea044bd58c8959cc80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-23T13:22:52.661 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:52 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:52.379+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:22:52.661 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:52 smithi007 podman[93939]: 2024-03-23 13:22:52.287917696 +0000 UTC m=+0.017349569 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:22:52.661 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:52 smithi007 podman[93939]: 2024-03-23 13:22:52.409692044 +0000 UTC m=+0.139123901 container create 47e1e46faa6e8c9457220ae9701853b2476fbfd42d4f8b2a08478f904849faea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:22:52.697 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:52 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:52.350+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:52.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:52.417+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:53.010 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:52 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:52.657+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:53.011 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:52 smithi007 podman[93939]: 2024-03-23 13:22:52.70424052 +0000 UTC m=+0.433672377 container init 47e1e46faa6e8c9457220ae9701853b2476fbfd42d4f8b2a08478f904849faea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-23T13:22:53.011 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:52 smithi007 podman[93939]: 2024-03-23 13:22:52.707376891 +0000 UTC m=+0.436808756 container start 47e1e46faa6e8c9457220ae9701853b2476fbfd42d4f8b2a08478f904849faea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:22:53.011 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:52 smithi007 podman[93939]: 2024-03-23 13:22:52.804366641 +0000 UTC m=+0.533798499 container attach 47e1e46faa6e8c9457220ae9701853b2476fbfd42d4f8b2a08478f904849faea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-23T13:22:53.290 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 23 13:22:53 smithi007 podman[93939]: 2024-03-23 13:22:53.008236125 +0000 UTC m=+0.737667987 container died 47e1e46faa6e8c9457220ae9701853b2476fbfd42d4f8b2a08478f904849faea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:22:53.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:52.940+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:53.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:53 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:53.336+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:22:53.698 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:53 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:53.313+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:53.766 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.3.service' 2024-03-23T13:22:53.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:53.456+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:53.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:22:53.806 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-23T13:22:53.807 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-23T13:22:53.807 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.4 2024-03-23T13:22:54.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:53 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:53.692+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:54.109 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:22:53 smithi130 systemd[1]: Stopping Ceph osd.4 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:22:54.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:53.971+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:54.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:54 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:54.286+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:54.449 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:22:54 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4[47820]: 2024-03-23T13:22:54.106+0000 7f119bbbb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:22:54.449 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:22:54 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4[47820]: 2024-03-23T13:22:54.106+0000 7f119bbbb700 -1 osd.4 90 *** Got signal Terminated *** 2024-03-23T13:22:54.449 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:22:54 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4[47820]: 2024-03-23T13:22:54.106+0000 7f119bbbb700 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:22:54.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:54 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:54.332+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:22:54.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:54.462+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:55.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:54 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:54.690+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:55.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:54.939+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:55.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:55 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:55.286+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:22:55.698 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:55 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:55.292+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:55.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:55.441+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:55.792 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:22:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:22:55.641+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:22:56.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:55 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:55.727+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:56.198 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:22:55 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:22:55.939+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:22:56.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:55.927+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:56.698 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:56 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:56.252+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:56.698 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:56 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:56.252+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:22:56.718 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:56 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:56.330+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:22:56.791 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:22:56 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:22:56.594+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:22:56.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:56 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:56.392+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:56.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:56 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:56.392+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:22:57.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:56 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:56.715+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:57.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:56 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:56.715+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:22:57.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:56 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:56.951+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:57.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:22:56 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:22:56.988+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:22:57.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:57 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:57.213+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:57.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:57 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:57.213+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:22:57.687 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:57 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:57.348+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:22:57.687 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:57 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:57.348+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:22:57.791 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:22:57 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:22:57.565+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:22:57.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:57 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:57.411+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:57.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:57 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:57.411+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:22:58.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:57 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:57.684+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:58.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:57 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:57.684+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:22:58.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:57 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:57.969+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:58.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:57 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:57.969+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.540337+0000 front 2024-03-23T13:22:31.540366+0000 (oldest deadline 2024-03-23T13:22:57.440389+0000) 2024-03-23T13:22:58.448 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:22:58 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4[47820]: 2024-03-23T13:22:58.217+0000 7f11981af700 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.914395+0000 front 2024-03-23T13:22:33.914450+0000 (oldest deadline 2024-03-23T13:22:58.014752+0000) 2024-03-23T13:22:58.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:22:58 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:22:58.018+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:22:58.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:58 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:58.201+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:58.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:58 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:58.201+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:22:58.681 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:58 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:58.349+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:22:58.682 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:58 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:58.349+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:22:58.791 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:22:58 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:22:58.546+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:22:58.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:58 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:58.388+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:58.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:58 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:58.388+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:22:59.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:58 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:58.677+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:22:59.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:58 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:58.677+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:22:59.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:58 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:58.940+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:22:59.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:58 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:58.940+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.540337+0000 front 2024-03-23T13:22:31.540366+0000 (oldest deadline 2024-03-23T13:22:57.440389+0000) 2024-03-23T13:22:59.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:22:59 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:22:59.043+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:22:59.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:59 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:59.213+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:22:59.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:22:59 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:22:59.213+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:22:59.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:59 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:59.314+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:22:59.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:22:59 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:22:59.314+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:22:59.791 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:22:59 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:22:59.569+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:22:59.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:59 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:59.427+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:22:59.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:22:59 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:22:59.427+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:22:59.869 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:22:59 smithi130 podman[70488]: 2024-03-23 13:22:59.514400089 +0000 UTC m=+5.515259891 container died dbb5c5e85203b4aa28dac58523761682e855b6e3b55de7252d1531858e1aa596 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:23:00.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:59 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:59.672+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:00.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:22:59 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:22:59.672+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:00.161 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:22:59 smithi130 podman[70488]: 2024-03-23 13:22:59.957123841 +0000 UTC m=+5.957983635 container cleanup dbb5c5e85203b4aa28dac58523761682e855b6e3b55de7252d1531858e1aa596 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-03-23T13:23:00.161 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:22:59 smithi130 bash[70488]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4 2024-03-23T13:23:00.161 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:00 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:00.064+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:00.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:59 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:59.930+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:23:00.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:22:59 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:22:59.930+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.540337+0000 front 2024-03-23T13:22:31.540366+0000 (oldest deadline 2024-03-23T13:22:57.440389+0000) 2024-03-23T13:23:00.448 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:23:00 smithi130 podman[70505]: 2024-03-23 13:23:00.158227786 +0000 UTC m=+0.637996614 container remove dbb5c5e85203b4aa28dac58523761682e855b6e3b55de7252d1531858e1aa596 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:23:00.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:00 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:00.197+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:00.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:00 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:00.197+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:00.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:00 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:00.297+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:00.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:00 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:00.297+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:00.791 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:00 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:00.561+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:00.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:00 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:00.403+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:00.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:00 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:00.403+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:00.906 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:23:00 smithi130 podman[70599]: 2024-03-23 13:23:00.473955588 +0000 UTC m=+0.017176052 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:23:00.906 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:23:00 smithi130 podman[70599]: 2024-03-23 13:23:00.583967309 +0000 UTC m=+0.127187771 container create d4813376bed1a89cda42feb3475ed453d4c2d23bd3aa20e7b6ca3227b4c623b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-23T13:23:00.906 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:23:00 smithi130 podman[70599]: 2024-03-23 13:23:00.811808666 +0000 UTC m=+0.355029144 container init d4813376bed1a89cda42feb3475ed453d4c2d23bd3aa20e7b6ca3227b4c623b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:23:00.906 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:23:00 smithi130 podman[70599]: 2024-03-23 13:23:00.81474595 +0000 UTC m=+0.357966420 container start d4813376bed1a89cda42feb3475ed453d4c2d23bd3aa20e7b6ca3227b4c623b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:23:00.907 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:23:00 smithi130 podman[70599]: 2024-03-23 13:23:00.903312794 +0000 UTC m=+0.446533275 container attach d4813376bed1a89cda42feb3475ed453d4c2d23bd3aa20e7b6ca3227b4c623b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-23T13:23:01.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:00 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:00.681+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:01.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:00 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:00.681+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:01.181 INFO:journalctl@ceph.osd.4.smithi130.stdout:Mar 23 13:23:01 smithi130 podman[70599]: 2024-03-23 13:23:01.110288289 +0000 UTC m=+0.653508753 container died d4813376bed1a89cda42feb3475ed453d4c2d23bd3aa20e7b6ca3227b4c623b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-23T13:23:01.181 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:01 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:01.085+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:01.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:00 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:00.925+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:23:01.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:00 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:00.925+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.540337+0000 front 2024-03-23T13:22:31.540366+0000 (oldest deadline 2024-03-23T13:22:57.440389+0000) 2024-03-23T13:23:01.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:01 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:01.178+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:01.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:01 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:01.178+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:01.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:01 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:01.259+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:01.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:01 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:01.259+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:01.661 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.4.service' 2024-03-23T13:23:01.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:23:01.703 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-23T13:23:01.703 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-23T13:23:01.703 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.5 2024-03-23T13:23:01.733 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:01 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:01.570+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:01.733 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:01 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:01.377+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:01.733 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:01 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:01.377+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:02.031 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:01 smithi155 systemd[1]: Stopping Ceph osd.5 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:23:02.032 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:01 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:01.923+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:23:02.032 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:01 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:01.923+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.540337+0000 front 2024-03-23T13:22:31.540366+0000 (oldest deadline 2024-03-23T13:22:57.440389+0000) 2024-03-23T13:23:02.032 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:02 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:02.027+0000 7fdc8f305700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:23:02.032 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:02 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:02.027+0000 7fdc8f305700 -1 osd.5 90 *** Got signal Terminated *** 2024-03-23T13:23:02.032 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:02 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:02.027+0000 7fdc8f305700 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:23:02.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:01 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:01.693+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:02.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:01 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:01.693+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:02.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:02 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:02.130+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:02.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:02 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:02.201+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:02.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:02 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:02.201+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:02.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:02 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:02.281+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:02.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:02 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:02.281+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:02.792 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:02 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:02.576+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:02.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:02 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:02.417+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:02.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:02 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:02.417+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:03.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:02 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:02.742+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:03.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:02 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:02.742+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:03.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:02 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:02.881+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:23:03.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:02 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:02.881+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.540337+0000 front 2024-03-23T13:22:31.540366+0000 (oldest deadline 2024-03-23T13:22:57.440389+0000) 2024-03-23T13:23:03.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:03 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:03.145+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:03.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:03 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:03.204+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:03.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:03 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:03.204+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:03.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:03 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:03.274+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:03.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:03 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:03.274+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:03.792 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:03 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:03.552+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:03.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:03 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:03.383+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:03.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:03 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:03.383+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:04.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:03 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:03.715+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:04.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:03 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:03.715+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:04.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:03 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:03.886+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:23:04.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:03 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:03.886+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.540337+0000 front 2024-03-23T13:22:31.540366+0000 (oldest deadline 2024-03-23T13:22:57.440389+0000) 2024-03-23T13:23:04.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:04 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:04.122+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:04.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:04 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:04.172+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:04.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:04 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:04.172+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:04.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:04 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:04.283+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:04.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:04 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:04.283+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:04.588 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:04 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:04.335+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:04.588 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:04 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:04.335+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:04.878 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:04 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:04.585+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:04.878 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:04 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:04.585+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:05.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:04 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:04.749+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:05.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:04 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:04.749+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:05.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:04 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:04.875+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:23:05.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:04 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:04.875+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.540337+0000 front 2024-03-23T13:22:31.540366+0000 (oldest deadline 2024-03-23T13:22:57.440389+0000) 2024-03-23T13:23:05.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:05 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:05.143+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:05.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:05 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:05.216+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:05.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:05 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:05.216+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:05.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:05 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:05.216+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:05.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:05 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:05.281+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:05.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:05 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:05.281+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:05.595 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:05 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:05.339+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:05.595 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:05 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:05.339+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:05.865 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:05 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:05.592+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:05.865 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:05 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:05.592+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:06.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:05 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:05.718+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:06.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:05 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:05.718+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:06.291 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:05 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:05.862+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:23:06.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:05 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:05.862+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.540337+0000 front 2024-03-23T13:22:31.540366+0000 (oldest deadline 2024-03-23T13:22:57.440389+0000) 2024-03-23T13:23:06.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:06 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:06.138+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:06.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:06 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:06.199+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:06.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:06 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:06.199+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:06.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:06 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:06.199+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:06.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:06 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:06.299+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:06.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:06 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:06.299+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:06.792 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:06 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:06.625+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:06.792 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:06 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:06.625+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:06.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:06 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:06.381+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:06.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:06 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:06.381+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:06.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:06 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:06.381+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:07.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:06 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:06.682+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:07.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:06 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:06.682+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:07.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:06 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:06.682+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.648953+0000 front 2024-03-23T13:22:41.648864+0000 (oldest deadline 2024-03-23T13:23:05.748770+0000) 2024-03-23T13:23:07.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:06 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:06.895+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:24.038546+0000 front 2024-03-23T13:22:24.038419+0000 (oldest deadline 2024-03-23T13:22:47.538575+0000) 2024-03-23T13:23:07.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:06 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5[48142]: 2024-03-23T13:23:06.895+0000 7fdc8b8f9700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.540337+0000 front 2024-03-23T13:22:31.540366+0000 (oldest deadline 2024-03-23T13:22:57.440389+0000) 2024-03-23T13:23:07.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:07 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:07.123+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:07.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:07 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:07.160+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:07.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:07 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:07.160+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:07.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:07 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:07.160+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:07.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:07 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:07.271+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:07.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:07 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:07.271+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:07.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:07 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:07.271+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:07.670 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:07 smithi155 podman[70651]: 2024-03-23 13:23:07.364926175 +0000 UTC m=+5.469331043 container died 96021899651f68fca62aee03efb347363879ac4051c09aefbf150b3f7103feb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-23T13:23:07.671 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:07 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:07.426+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:07.671 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:07 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:07.426+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:07.671 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:07 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:07.426+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:08.040 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:07 smithi155 podman[70651]: 2024-03-23 13:23:07.794870545 +0000 UTC m=+5.899275399 container cleanup 96021899651f68fca62aee03efb347363879ac4051c09aefbf150b3f7103feb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-03-23T13:23:08.041 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:07 smithi155 bash[70651]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5 2024-03-23T13:23:08.041 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:07 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:07.667+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:08.041 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:07 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:07.667+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:08.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:07 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:07.702+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:08.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:07 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:07.702+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:08.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:07 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:07.702+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.648953+0000 front 2024-03-23T13:22:41.648864+0000 (oldest deadline 2024-03-23T13:23:05.748770+0000) 2024-03-23T13:23:08.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:08 smithi155 podman[70669]: 2024-03-23 13:23:08.037877558 +0000 UTC m=+0.667312236 container remove 96021899651f68fca62aee03efb347363879ac4051c09aefbf150b3f7103feb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-03-23T13:23:08.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:08 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:08.157+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:08.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:08 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:08.143+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:08.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:08 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:08.143+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:08.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:08 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:08.143+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:08.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:08 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:08.311+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:08.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:08 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:08.311+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:08.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:08 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:08.311+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:08.701 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:08 smithi155 podman[70763]: 2024-03-23 13:23:08.358374343 +0000 UTC m=+0.017745051 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:23:08.702 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:08 smithi155 podman[70763]: 2024-03-23 13:23:08.472410375 +0000 UTC m=+0.131781089 container create 81c89c1d34087f5566543904b6b205a4099402227d9ab2752870d76f05026c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=) 2024-03-23T13:23:08.702 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:08 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:08.377+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:08.702 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:08 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:08.377+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:08.702 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:08 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:08.377+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:08.999 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:08 smithi155 podman[70763]: 2024-03-23 13:23:08.699667519 +0000 UTC m=+0.359038215 container init 81c89c1d34087f5566543904b6b205a4099402227d9ab2752870d76f05026c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS) 2024-03-23T13:23:09.000 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:08 smithi155 podman[70763]: 2024-03-23 13:23:08.702856169 +0000 UTC m=+0.362226868 container start 81c89c1d34087f5566543904b6b205a4099402227d9ab2752870d76f05026c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux ) 2024-03-23T13:23:09.000 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:08 smithi155 podman[70763]: 2024-03-23 13:23:08.808091307 +0000 UTC m=+0.467462024 container attach 81c89c1d34087f5566543904b6b205a4099402227d9ab2752870d76f05026c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2) 2024-03-23T13:23:09.000 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:08 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:08.698+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:09.000 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:08 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:08.698+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:09.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:08 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:08.722+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:09.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:08 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:08.722+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:09.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:08 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:08.722+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.648953+0000 front 2024-03-23T13:22:41.648864+0000 (oldest deadline 2024-03-23T13:23:05.748770+0000) 2024-03-23T13:23:09.292 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 23 13:23:08 smithi155 podman[70763]: 2024-03-23 13:23:08.996503482 +0000 UTC m=+0.655874177 container died 81c89c1d34087f5566543904b6b205a4099402227d9ab2752870d76f05026c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:23:09.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:09.108+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:09.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:09.108+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:09.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:09.148+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:09.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:09.148+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:09.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:09 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:09.148+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:09.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:09.310+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:09.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:09.310+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:09.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:09.310+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:09.597 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:09.404+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:09.598 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:09.404+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:09.598 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:09.404+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:09.613 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.5.service' 2024-03-23T13:23:09.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:23:09.693 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-23T13:23:09.694 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-23T13:23:09.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.6 2024-03-23T13:23:10.014 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:09.695+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:10.015 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:09.695+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:10.015 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:09 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:09.695+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.648953+0000 front 2024-03-23T13:22:41.648864+0000 (oldest deadline 2024-03-23T13:23:05.748770+0000) 2024-03-23T13:23:10.015 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:09 smithi007 systemd[1]: Stopping Ceph osd.6 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:23:10.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:09.691+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:10.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:09 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:09.691+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:10.295 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:10.010+0000 7f23a41ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:23:10.295 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:10.010+0000 7f23a41ca700 -1 osd.6 90 *** Got signal Terminated *** 2024-03-23T13:23:10.296 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:10.010+0000 7f23a41ca700 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:23:10.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:10 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:10.119+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:10.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:10 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:10.119+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:10.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:10 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:10.165+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:10.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:10 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:10.165+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:10.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:10 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:10.165+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:10.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:10.291+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:10.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:10.291+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:10.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:10.291+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:10.710 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:10 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:10.399+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:10.711 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:10 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:10.399+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:10.711 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:10 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:10.399+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:11.041 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:10 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:10.707+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:11.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:10 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:10.707+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:11.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:10.653+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:11.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:10.653+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:11.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:10.653+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.648953+0000 front 2024-03-23T13:22:41.648864+0000 (oldest deadline 2024-03-23T13:23:05.748770+0000) 2024-03-23T13:23:11.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:11 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:11.109+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:11.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:11 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:11.109+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:11.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:11 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:11.140+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:11.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:11 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:11.140+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:11.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:11 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:11.140+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:11.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:11 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:11.274+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:11.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:11 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:11.274+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:11.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:11 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:11.274+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:11.693 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:11 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:11.358+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:11.693 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:11 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:11.358+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:11.693 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:11 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:11.358+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:12.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:11 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:11.689+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:12.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:11 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:11.689+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:12.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:11 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:11.637+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:12.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:11 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:11.637+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:12.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:11 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:11.637+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.648953+0000 front 2024-03-23T13:22:41.648864+0000 (oldest deadline 2024-03-23T13:23:05.748770+0000) 2024-03-23T13:23:12.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:12 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:12.084+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:12.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:12 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:12.084+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:12.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:12 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:12.118+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:12.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:12 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:12.118+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:12.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:12 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:12.118+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:12.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:12.281+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:12.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:12.281+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:12.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:12.281+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:12.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:12.281+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:12.692 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:12 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:12.350+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:12.693 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:12 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:12.351+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:12.693 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:12 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:12.351+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:13.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:12 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:12.689+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:13.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:12 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:12.689+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:13.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:12.634+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:13.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:12.634+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:13.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:12 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:12.634+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.648953+0000 front 2024-03-23T13:22:41.648864+0000 (oldest deadline 2024-03-23T13:23:05.748770+0000) 2024-03-23T13:23:13.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:13 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:13.117+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:13.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:13 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:13.117+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:13.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:13 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:13.117+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:13.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:13 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:13.080+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:13.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:13 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:13.080+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:13.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:13 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:13.080+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:13.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:13 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:13.080+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:13.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:13 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:13.313+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:13.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:13 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:13.313+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:13.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:13 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:13.313+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:13.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:13 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:13.313+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:13.737 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:13 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:13.327+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:13.737 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:13 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:13.327+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:13.737 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:13 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:13.327+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:14.041 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:13 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:13.733+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:14.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:13 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:13.733+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:14.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:13 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:13.733+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:14.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:13 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:13.590+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:14.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:13 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:13.590+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:14.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:13 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:13.590+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.648953+0000 front 2024-03-23T13:22:41.648864+0000 (oldest deadline 2024-03-23T13:23:05.748770+0000) 2024-03-23T13:23:14.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:13 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:13.590+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:46.849965+0000 front 2024-03-23T13:22:46.849706+0000 (oldest deadline 2024-03-23T13:23:12.650234+0000) 2024-03-23T13:23:14.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:14 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:14.104+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:14.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:14 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:14.104+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:14.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:14 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:14.104+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:14.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:14 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:14.051+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:14.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:14 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:14.051+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:14.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:14 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:14.051+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:14.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:14 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:14.051+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:14.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:14.324+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:14.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:14.324+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:14.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:14.324+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:14.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:14.324+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:14.699 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:14 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:14.328+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:14.699 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:14 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:14.328+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:14.700 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:14 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:14.328+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:15.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:14 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:14.696+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:15.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:14 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:14.696+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:15.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:14 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:14.696+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:15.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:14.599+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.445147+0000 front 2024-03-23T13:22:25.445172+0000 (oldest deadline 2024-03-23T13:22:47.145337+0000) 2024-03-23T13:23:15.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:14.599+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.847713+0000 front 2024-03-23T13:22:32.847517+0000 (oldest deadline 2024-03-23T13:22:56.347582+0000) 2024-03-23T13:23:15.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:14.599+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.648953+0000 front 2024-03-23T13:22:41.648864+0000 (oldest deadline 2024-03-23T13:23:05.748770+0000) 2024-03-23T13:23:15.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:14 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6[60323]: 2024-03-23T13:23:14.599+0000 7f239c7b6700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:46.849965+0000 front 2024-03-23T13:22:46.849706+0000 (oldest deadline 2024-03-23T13:23:12.650234+0000) 2024-03-23T13:23:15.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:15 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:15.063+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:15.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:15 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:15.063+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:15.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:15 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:15.063+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:15.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:15 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:15.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:15.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:15 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:15.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:15.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:15 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:15.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:15.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:15 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:15.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:15.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:15 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:15.283+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:15.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:15 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:15.283+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:15.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:15 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:15.283+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:15.580 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:15 smithi007 podman[94056]: 2024-03-23 13:23:15.313109207 +0000 UTC m=+5.425932689 container died 0c3a434d12f7905f1834f319e0611aa35d51927797d4011d02ee4b92de149681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:23:15.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:15 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:15.335+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:15.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:15 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:15.335+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:15.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:15 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:15.335+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:15.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:15 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:15.335+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:16.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:15 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:15.721+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:16.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:15 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:15.721+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:16.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:15 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:15.721+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:16.080 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:15 smithi007 podman[94056]: 2024-03-23 13:23:15.789196923 +0000 UTC m=+5.902020404 container cleanup 0c3a434d12f7905f1834f319e0611aa35d51927797d4011d02ee4b92de149681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-23T13:23:16.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:15 smithi007 bash[94056]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6 2024-03-23T13:23:16.417 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:16 smithi007 podman[94073]: 2024-03-23 13:23:16.097144142 +0000 UTC m=+0.777262893 container remove 0c3a434d12f7905f1834f319e0611aa35d51927797d4011d02ee4b92de149681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=) 2024-03-23T13:23:16.417 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:16 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:16.287+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:16.417 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:16 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:16.287+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:16.418 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:16 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:16.287+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:16.418 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:16 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:16.287+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:16.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:16 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:16.068+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:16.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:16 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:16.068+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:16.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:16 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:16.068+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:16.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:16 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:16.111+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:16.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:16 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:16.111+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:16.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:16 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:16.111+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:16.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:16 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:16.111+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:16.740 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:16 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:16.306+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:16.740 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:16 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:16.306+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:16.740 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:16 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:16.306+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:16.830 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:16 smithi007 podman[94162]: 2024-03-23 13:23:16.317001056 +0000 UTC m=+0.017606258 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:23:16.830 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:16 smithi007 podman[94162]: 2024-03-23 13:23:16.416666864 +0000 UTC m=+0.117272066 container create 16e2c5e0162952dda930243844020a88461dcd2aeb96b5344c1b99b207fbc45e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:23:17.041 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:16 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:16.736+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:17.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:16 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:16.736+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:17.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:16 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:16.736+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:17.234 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:16 smithi007 podman[94162]: 2024-03-23 13:23:16.929940268 +0000 UTC m=+0.630545465 container init 16e2c5e0162952dda930243844020a88461dcd2aeb96b5344c1b99b207fbc45e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-23T13:23:17.234 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:16 smithi007 podman[94162]: 2024-03-23 13:23:16.93342864 +0000 UTC m=+0.634033838 container start 16e2c5e0162952dda930243844020a88461dcd2aeb96b5344c1b99b207fbc45e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-23T13:23:17.234 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:17 smithi007 podman[94162]: 2024-03-23 13:23:17.019394249 +0000 UTC m=+0.719999454 container attach 16e2c5e0162952dda930243844020a88461dcd2aeb96b5344c1b99b207fbc45e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-23T13:23:17.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:17 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:17.077+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:17.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:17 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:17.077+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:17.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:17 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:17.077+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:17.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:17 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:17.120+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:17.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:17 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:17.120+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:17.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:17 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:17.120+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:17.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:17 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:17.120+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:17.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:17 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:17.278+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:17.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:17 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:17.278+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:17.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:17 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:17.278+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:17.543 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 23 13:23:17 smithi007 podman[94162]: 2024-03-23 13:23:17.232093784 +0000 UTC m=+0.932698983 container died 16e2c5e0162952dda930243844020a88461dcd2aeb96b5344c1b99b207fbc45e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-23T13:23:17.543 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:17 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:17.275+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:17.543 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:17 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:17.275+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:17.543 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:17 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:17.275+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:17.543 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:17 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:17.275+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:17.942 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.6.service' 2024-03-23T13:23:17.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:23:17.989 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-23T13:23:17.990 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-23T13:23:17.990 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.7 2024-03-23T13:23:18.041 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:17 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:17.738+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:18.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:17 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:17.738+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:18.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:17 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:17.738+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:18.296 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:18 smithi130 systemd[1]: Stopping Ceph osd.7 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:23:18.297 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:18 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:18.046+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:18.297 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:18 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:18.046+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:18.297 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:18 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:18.046+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:18.297 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:18 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:18.134+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:18.297 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:18 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:18.134+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:18.297 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:18 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:18.134+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:18.298 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:18 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:18.134+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:18.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:18 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:18.242+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:18.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:18 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:18.242+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:18.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:18 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:18.242+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:18.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:18 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:18.242+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:18.698 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:18 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:18.292+0000 7efd32323700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:23:18.698 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:18 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:18.292+0000 7efd32323700 -1 osd.7 90 *** Got signal Terminated *** 2024-03-23T13:23:18.698 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:18 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:18.292+0000 7efd32323700 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:23:18.735 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:18 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:18.309+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:18.735 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:18 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:18.309+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:18.735 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:18 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:18.309+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:19.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:18 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:18.732+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:19.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:18 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:18.732+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:19.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:18 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:18.732+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:19.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:19 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:19.025+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:19.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:19 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:19.025+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:19.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:19 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:19.025+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:19.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:19 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:19.129+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:19.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:19 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:19.129+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:19.450 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:19 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:19.129+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:19.450 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:19 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:19.129+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:19.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:19 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:19.264+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:19.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:19 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:19.264+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:19.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:19 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:19.264+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:19.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:19.270+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:19.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:19.270+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:19.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:19.270+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:19.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:19 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:19.270+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:19.816 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:19 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:19.724+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:19.817 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:19 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:19.725+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:19.817 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:19 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:19.725+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:20.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:20 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:20.142+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:20.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:20 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:20.142+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:20.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:20 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:20.142+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:20.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:20 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:20.142+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:20.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:20 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:20.009+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:20.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:20 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:20.009+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:20.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:20 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:20.009+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:20.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:20 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:20.247+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:20.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:20 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:20.247+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:20.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:20 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:20.247+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:20.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:20 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:20.247+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:20.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:20 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:20.292+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:20.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:20 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:20.292+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:20.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:20 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:20.292+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:20.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:20 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:20.292+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:21.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:20 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:20.690+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:21.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:20 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:20.690+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:21.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:20 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:20.690+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:21.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:20 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:20.690+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:21.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:21 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:21.049+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:21.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:21 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:21.049+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:21.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:21 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:21.049+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:21.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:21 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:21.188+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:21.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:21 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:21.188+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:21.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:21 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:21.188+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:21.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:21 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:21.188+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:21.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:21 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:21.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:21.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:21 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:21.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:21.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:21 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:21.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:21.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:21 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:21.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:21.715 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:21 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:21.291+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:21.715 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:21 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:21.291+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:21.715 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:21 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:21.291+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:21.715 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:21 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:21.291+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:22.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:21 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:21.712+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:22.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:21 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:21.712+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:22.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:21 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:21.712+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:22.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:21 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:21.712+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:22.448 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:22 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:22.053+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:22.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:22 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:22.053+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:22.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:22 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:22.053+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:22.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:22 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:22.171+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:22.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:22 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:22.171+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:22.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:22 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:22.171+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:22.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:22 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:22.171+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:22.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:22 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:22.260+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:22.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:22 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:22.260+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:22.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:22 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:22.260+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:22.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:22 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:22.260+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:22.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:22 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:22.277+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:22.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:22 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:22.277+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:22.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:22 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:22.277+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:22.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:22 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:22.277+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:22.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:22 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:22.277+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:23.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:22 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:22.695+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:23.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:22 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:22.695+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:23.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:22 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:22.695+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:23.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:22 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:22.695+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:23.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:23 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:23.157+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:23.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:23 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:23.157+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:23.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:23 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:23.157+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:23.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:23 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:23.157+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:23.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:23 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:23.090+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.029363+0000 front 2024-03-23T13:22:31.029590+0000 (oldest deadline 2024-03-23T13:22:55.729570+0000) 2024-03-23T13:23:23.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:23 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:23.090+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:38.630547+0000 front 2024-03-23T13:22:38.630680+0000 (oldest deadline 2024-03-23T13:23:08.631085+0000) 2024-03-23T13:23:23.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:23 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:23.090+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:50.332122+0000 front 2024-03-23T13:22:50.332313+0000 (oldest deadline 2024-03-23T13:23:12.632323+0000) 2024-03-23T13:23:23.449 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:23 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7[54850]: 2024-03-23T13:23:23.090+0000 7efd2a90f700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:56.633999+0000 front 2024-03-23T13:22:56.634107+0000 (oldest deadline 2024-03-23T13:23:22.534136+0000) 2024-03-23T13:23:23.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:23 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:23.240+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:23.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:23 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:23.240+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:23.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:23 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:23.240+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:23.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:23 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:23.240+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:23.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:23 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:23.240+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:23.658 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:23 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:23.302+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:23.658 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:23 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:23.302+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:23.658 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:23 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:23.302+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:23.658 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:23 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:23.302+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:24.031 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:23 smithi130 podman[70717]: 2024-03-23 13:23:23.695271577 +0000 UTC m=+5.512340876 container died 37eab996ac174fcf0da14ef6408902a4c3374b54ecc7f5dc69302088f49e0fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-23T13:23:24.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:23 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:23.654+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:24.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:23 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:23.654+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:24.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:23 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:23.654+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:24.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:23 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:23.654+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:24.380 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:24 smithi130 podman[70717]: 2024-03-23 13:23:24.126603388 +0000 UTC m=+5.943672694 container cleanup 37eab996ac174fcf0da14ef6408902a4c3374b54ecc7f5dc69302088f49e0fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-23T13:23:24.381 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:24 smithi130 bash[70717]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7 2024-03-23T13:23:24.381 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:24 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:24.204+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:24.381 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:24 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:24.204+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:24.381 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:24 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:24.204+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:24.382 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:24 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:24.204+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:24.382 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:24 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:24.204+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:24.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:24 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:24.271+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:24.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:24 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:24.271+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:24.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:24 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:24.271+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:24.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:24 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:24.271+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:24.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:24 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:24.271+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:24.698 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:24 smithi130 podman[70734]: 2024-03-23 13:23:24.377801304 +0000 UTC m=+0.676805276 container remove 37eab996ac174fcf0da14ef6408902a4c3374b54ecc7f5dc69302088f49e0fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-23T13:23:24.705 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:24 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:24.315+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:24.706 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:24 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:24.315+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:24.706 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:24 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:24.315+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:24.706 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:24 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:24.315+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:25.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:24 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:24.701+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:25.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:24 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:24.701+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:25.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:24 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:24.701+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:25.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:24 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:24.702+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:25.043 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:24 smithi130 podman[70824]: 2024-03-23 13:23:24.687837241 +0000 UTC m=+0.017602000 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:23:25.043 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:24 smithi130 podman[70824]: 2024-03-23 13:23:24.795284283 +0000 UTC m=+0.125049022 container create 58f6e4562df9ad2ee0247ba8d78258b9ec8c342a6803de313d9be574ff47fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:23:25.043 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:25 smithi130 podman[70824]: 2024-03-23 13:23:25.039198758 +0000 UTC m=+0.368963516 container init 58f6e4562df9ad2ee0247ba8d78258b9ec8c342a6803de313d9be574ff47fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-23T13:23:25.320 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:25 smithi130 podman[70824]: 2024-03-23 13:23:25.042742431 +0000 UTC m=+0.372507182 container start 58f6e4562df9ad2ee0247ba8d78258b9ec8c342a6803de313d9be574ff47fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-23T13:23:25.320 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:25 smithi130 podman[70824]: 2024-03-23 13:23:25.130919431 +0000 UTC m=+0.460684206 container attach 58f6e4562df9ad2ee0247ba8d78258b9ec8c342a6803de313d9be574ff47fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:23:25.320 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:25 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:25.164+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:25.320 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:25 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:25.164+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:25.320 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:25 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:25.164+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:25.321 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:25 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:25.164+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:25.321 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:25 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:25.164+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:25.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:25 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:25.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:25.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:25 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:25.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:25.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:25 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:25.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:25.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:25 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:25.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:25.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:25 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:25.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:25.629 INFO:journalctl@ceph.osd.7.smithi130.stdout:Mar 23 13:23:25 smithi130 podman[70824]: 2024-03-23 13:23:25.340953616 +0000 UTC m=+0.670718375 container died 58f6e4562df9ad2ee0247ba8d78258b9ec8c342a6803de313d9be574ff47fe61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:23:25.677 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:25 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:25.291+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:25.677 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:25 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:25.291+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:25.677 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:25 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:25.291+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:25.678 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:25 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:25.291+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:25.971 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.7.service' 2024-03-23T13:23:26.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:23:26.012 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-23T13:23:26.012 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-23T13:23:26.013 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.8 2024-03-23T13:23:26.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:25 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:25.673+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:26.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:25 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:25.673+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:26.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:25 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:25.673+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:26.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:25 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:25.673+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:26.198 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:26 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:26.151+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:26.199 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:26 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:26.151+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:26.199 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:26 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:26.151+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:26.199 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:26 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:26.151+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:26.199 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:26 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:26.151+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:26.337 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:26 smithi155 systemd[1]: Stopping Ceph osd.8 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:23:26.580 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:26 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:26.309+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:26.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:26 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:26.309+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:26.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:26 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:26.309+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:26.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:26 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:26.309+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:26.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:26 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:26.309+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:26.662 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:26.338+0000 7f2f68fc5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:23:26.663 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:26.338+0000 7f2f68fc5700 -1 osd.8 90 *** Got signal Terminated *** 2024-03-23T13:23:26.663 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:26.338+0000 7f2f68fc5700 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:23:26.663 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:26.658+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:26.663 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:26.658+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:26.663 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:26.658+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:26.663 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:26.658+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:26.664 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:26.333+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:26.664 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:26.333+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:26.664 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:26.333+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:26.664 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:26 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:26.333+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:27.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:27 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:27.135+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:27.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:27 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:27.135+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:27.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:27 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:27.135+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:27.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:27 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:27.135+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:27.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:27 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:27.135+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:27.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:27 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:27.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:27.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:27 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:27.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:27.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:27 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:27.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:27.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:27 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:27.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:27.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:27 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:27.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:27.704 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:27 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:27.297+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:27.704 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:27 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:27.297+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:27.705 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:27 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:27.297+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:27.705 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:27 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:27.297+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:28.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:27 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:27.700+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:28.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:27 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:27.700+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:28.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:27 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:27.700+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:28.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:27 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:27.700+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:28.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:28 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:28.101+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:28.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:28 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:28.101+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:28.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:28 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:28.101+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:28.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:28 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:28.101+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:28.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:28 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:28.101+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:28.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:28 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:28.285+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:28.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:28 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:28.285+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:28.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:28 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:28.285+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:28.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:28 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:28.285+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:28.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:28 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:28.285+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:28.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:28 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:28.285+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:28.735 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:28 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:28.305+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:28.735 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:28 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:28.305+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:28.736 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:28 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:28.305+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:28.736 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:28 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:28.305+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:29.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:28 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:28.731+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:29.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:28 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:28.731+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:29.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:28 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:28.731+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:29.043 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:28 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:28.731+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:29.043 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:28 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:28.731+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.941826+0000 front 2024-03-23T13:23:03.941880+0000 (oldest deadline 2024-03-23T13:23:28.042106+0000) 2024-03-23T13:23:29.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:29 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:29.070+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:29.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:29 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:29.070+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:29.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:29 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:29.070+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:29.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:29 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:29.070+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:29.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:29 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:29.070+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:29.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:29 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:29.319+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:29.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:29 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:29.319+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:29.582 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:29 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:29.319+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:29.582 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:29 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:29.319+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:29.582 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:29 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:29.319+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:29.582 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:29 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:29.319+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:29.782 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:29 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:29.326+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:29.782 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:29 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:29.326+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:29.782 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:29 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:29.326+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:29.782 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:29 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:29.326+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:30.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:29 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:29.778+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:30.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:29 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:29.778+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:30.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:29 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:29.778+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:30.043 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:29 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:29.778+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:30.043 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:29 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:29.778+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.941826+0000 front 2024-03-23T13:23:03.941880+0000 (oldest deadline 2024-03-23T13:23:28.042106+0000) 2024-03-23T13:23:30.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:30 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:30.037+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:30.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:30 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:30.037+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:30.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:30 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:30.037+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:30.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:30 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:30.037+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:30.450 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:30 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:30.037+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:30.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:30.322+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:30.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:30.322+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:30.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:30.322+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:30.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:30.322+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:30.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:30.322+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:30.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:30 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:30.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:30.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:30 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:30.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:30.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:30 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:30.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:30.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:30 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:30.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:30.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:30 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:30.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:30.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:30 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:30.318+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:31.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:30.764+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.835819+0000 front 2024-03-23T13:22:33.835655+0000 (oldest deadline 2024-03-23T13:22:55.535559+0000) 2024-03-23T13:23:31.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:30.764+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:42.437919+0000 front 2024-03-23T13:22:42.437975+0000 (oldest deadline 2024-03-23T13:23:04.138051+0000) 2024-03-23T13:23:31.042 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:30.764+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:48.239330+0000 front 2024-03-23T13:22:48.239117+0000 (oldest deadline 2024-03-23T13:23:12.939195+0000) 2024-03-23T13:23:31.043 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:30.764+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.740958+0000 front 2024-03-23T13:22:58.740975+0000 (oldest deadline 2024-03-23T13:23:20.440911+0000) 2024-03-23T13:23:31.043 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8[54858]: 2024-03-23T13:23:30.764+0000 7f2f655b9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.941826+0000 front 2024-03-23T13:23:03.941880+0000 (oldest deadline 2024-03-23T13:23:28.042106+0000) 2024-03-23T13:23:31.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:31 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:31.033+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:31.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:31 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:31.033+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:31.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:31 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:31.033+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:31.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:31 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:31.033+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:31.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:31 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:31.033+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:31.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:31 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:31.290+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:31.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:31 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:31.290+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:31.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:31 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:31.290+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:31.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:31 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:31.290+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:31.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:31 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:31.290+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:31.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:31 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:31.353+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:31.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:31 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:31.353+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:31.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:31 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:31.353+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:31.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:31 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:31.353+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:31.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:31 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:31.353+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:31.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:31 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:31.353+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:32.143 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:31 smithi155 podman[70880]: 2024-03-23 13:23:31.848257666 +0000 UTC m=+5.640569605 container died 3200724919af8fa20904ea14cf741dee6fe9084f0b7c64005416e1a18f5c7016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:23:32.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:32 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:32.001+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:32.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:32 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:32.001+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:32.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:32 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:32.001+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:32.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:32 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:32.001+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:32.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:32 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:32.001+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:32.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:32 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:32.001+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:32.535 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:32 smithi155 podman[70880]: 2024-03-23 13:23:32.230712909 +0000 UTC m=+6.023024847 container cleanup 3200724919af8fa20904ea14cf741dee6fe9084f0b7c64005416e1a18f5c7016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-23T13:23:32.535 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:32 smithi155 bash[70880]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8 2024-03-23T13:23:32.535 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:32 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:32.314+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:32.536 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:32 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:32.314+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:32.536 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:32 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:32.314+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:32.536 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:32 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:32.314+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:32.536 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:32 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:32.314+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:32.792 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:32 smithi155 podman[70899]: 2024-03-23 13:23:32.531800859 +0000 UTC m=+0.674005372 container remove 3200724919af8fa20904ea14cf741dee6fe9084f0b7c64005416e1a18f5c7016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-23T13:23:32.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:32 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:32.363+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:32.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:32 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:32.363+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:32.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:32 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:32.363+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:32.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:32 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:32.363+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:32.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:32 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:32.363+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:32.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:32 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:32.363+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:33.138 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:32 smithi155 podman[70996]: 2024-03-23 13:23:32.74599071 +0000 UTC m=+0.018057404 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:23:33.138 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:32 smithi155 podman[70996]: 2024-03-23 13:23:32.866196965 +0000 UTC m=+0.138263714 container create 0c1ebaf05e16375970cfcad3ccd64a28f92c140578e0808283f8e805ba221e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-23T13:23:33.431 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:33 smithi155 podman[70996]: 2024-03-23 13:23:33.135520083 +0000 UTC m=+0.407586767 container init 0c1ebaf05e16375970cfcad3ccd64a28f92c140578e0808283f8e805ba221e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-23T13:23:33.431 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:33 smithi155 podman[70996]: 2024-03-23 13:23:33.138758116 +0000 UTC m=+0.410824810 container start 0c1ebaf05e16375970cfcad3ccd64a28f92c140578e0808283f8e805ba221e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:23:33.431 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:33 smithi155 podman[70996]: 2024-03-23 13:23:33.227056867 +0000 UTC m=+0.499123577 container attach 0c1ebaf05e16375970cfcad3ccd64a28f92c140578e0808283f8e805ba221e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:23:33.432 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:33 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:33.281+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:33.432 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:33 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:33.281+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:33.432 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:33 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:33.281+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:33.432 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:33 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:33.281+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:33.432 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:33 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:33.281+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:33.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:33 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:33.027+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:33.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:33 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:33.027+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:33.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:33 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:33.027+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:33.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:33 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:33.027+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:33.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:33 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:33.027+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:33.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:33 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:33.027+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:33.774 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 23 13:23:33 smithi155 podman[70996]: 2024-03-23 13:23:33.428060584 +0000 UTC m=+0.700127286 container died 0c1ebaf05e16375970cfcad3ccd64a28f92c140578e0808283f8e805ba221e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:23:33.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:33 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:33.389+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:33.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:33 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:33.389+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:33.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:33 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:33.389+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:33.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:33 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:33.389+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:33.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:33 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:33.389+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:33.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:33 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:33.389+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:34.061 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:34 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:34.055+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:34.061 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:34 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:34.055+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:34.061 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:34 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:34.055+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:34.061 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:34 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:34.055+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:34.061 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:34 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:34.055+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:34.062 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:34 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:34.055+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:34.103 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.8.service' 2024-03-23T13:23:34.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:23:34.184 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-23T13:23:34.185 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-23T13:23:34.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.9 2024-03-23T13:23:34.504 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:34 smithi007 systemd[1]: Stopping Ceph osd.9 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:23:34.505 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:34.423+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:34.505 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:34.423+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:34.505 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:34.423+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:34.505 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:34.423+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:34.505 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:34.423+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:34.505 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:34.423+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:34.505 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:34.500+0000 7f05cbbaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:23:34.506 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:34.500+0000 7f05cbbaf700 -1 osd.9 90 *** Got signal Terminated *** 2024-03-23T13:23:34.506 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:34 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:34.500+0000 7f05cbbaf700 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:23:34.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:34 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:34.331+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:34.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:34 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:34.331+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:34.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:34 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:34.331+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:34.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:34 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:34.331+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:34.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:34 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:34.331+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:35.350 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:35 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:35.022+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:35.350 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:35 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:35.022+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:35.351 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:35 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:35.022+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:35.351 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:35 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:35.022+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:35.351 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:35 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:35.022+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:35.351 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:35 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:35.022+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:35.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:35 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:35.338+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:35.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:35 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:35.338+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:35.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:35 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:35.338+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:35.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:35 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:35.338+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:35.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:35 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:35.338+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:35.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:35 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:35.378+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:35.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:35 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:35.378+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:35.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:35 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:35.378+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:35.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:35 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:35.378+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:35.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:35 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:35.378+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:35.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:35 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:35.378+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:36.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:36 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:36.057+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:36.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:36 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:36.057+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:36.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:36 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:36.057+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:36.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:36 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:36.057+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:36.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:36 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:36.057+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:36.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:36 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:36.057+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:36.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:36 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:36.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:36.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:36 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:36.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:36.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:36 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:36.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:36.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:36 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:36.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:36.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:36 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:36.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:36.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:36.345+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:36.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:36.345+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:36.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:36.345+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:36.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:36.345+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:36.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:36.345+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:36.832 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:36.345+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:36.832 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:36 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:36.345+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:12.370423+0000 front 2024-03-23T13:23:12.370460+0000 (oldest deadline 2024-03-23T13:23:35.870389+0000) 2024-03-23T13:23:37.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:37 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:37.028+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:37.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:37 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:37.028+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:37.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:37 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:37.028+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:37.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:37 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:37.028+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:37.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:37 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:37.028+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:37.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:37 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:37.028+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:37.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:37 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:37.253+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:37.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:37 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:37.253+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:37.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:37 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:37.253+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:37.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:37 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:37.253+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:37.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:37 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:37.253+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:37.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:37 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:37.380+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:37.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:37 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:37.380+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:37.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:37 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:37.380+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:37.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:37 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:37.380+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:37.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:37 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:37.380+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:37.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:37 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:37.380+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:37.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:37 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:37.380+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:12.370423+0000 front 2024-03-23T13:23:12.370460+0000 (oldest deadline 2024-03-23T13:23:35.870389+0000) 2024-03-23T13:23:38.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:38 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:38.030+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:38.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:38 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:38.030+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:38.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:38 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:38.030+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:38.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:38 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:38.030+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:38.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:38 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:38.030+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:38.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:38 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:38.030+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:38.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:38 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:38.030+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:13.679969+0000 front 2024-03-23T13:23:13.680051+0000 (oldest deadline 2024-03-23T13:23:37.779926+0000) 2024-03-23T13:23:38.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:38 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:38.210+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:38.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:38 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:38.210+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:38.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:38 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:38.210+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:38.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:38 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:38.210+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:38.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:38 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:38.210+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:38.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:38.332+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:38.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:38.332+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:38.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:38.332+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:38.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:38.332+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:38.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:38.332+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:38.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:38.332+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:38.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:38 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:38.332+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:12.370423+0000 front 2024-03-23T13:23:12.370460+0000 (oldest deadline 2024-03-23T13:23:35.870389+0000) 2024-03-23T13:23:39.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.023+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:39.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.023+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:39.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.023+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:39.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.023+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:39.450 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.023+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:39.450 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.023+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:39.450 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.023+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:13.679969+0000 front 2024-03-23T13:23:13.680051+0000 (oldest deadline 2024-03-23T13:23:37.779926+0000) 2024-03-23T13:23:39.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:39 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:39.185+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:39.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:39 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:39.185+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:39.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:39 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:39.185+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:39.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:39 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:39.185+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:39.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:39 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:39.185+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:39.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:39.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:25.362451+0000 front 2024-03-23T13:22:25.362420+0000 (oldest deadline 2024-03-23T13:22:50.062445+0000) 2024-03-23T13:23:39.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:39.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:33.464133+0000 front 2024-03-23T13:22:33.464101+0000 (oldest deadline 2024-03-23T13:22:56.364250+0000) 2024-03-23T13:23:39.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:39.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:41.065457+0000 front 2024-03-23T13:22:41.065430+0000 (oldest deadline 2024-03-23T13:23:06.365487+0000) 2024-03-23T13:23:39.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:39.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.466207+0000 front 2024-03-23T13:22:47.466281+0000 (oldest deadline 2024-03-23T13:23:11.467006+0000) 2024-03-23T13:23:39.581 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:39.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:59.068478+0000 front 2024-03-23T13:22:59.068457+0000 (oldest deadline 2024-03-23T13:23:21.968400+0000) 2024-03-23T13:23:39.582 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:39.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:03.069433+0000 front 2024-03-23T13:23:03.069498+0000 (oldest deadline 2024-03-23T13:23:27.769275+0000) 2024-03-23T13:23:39.582 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:39 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9[66913]: 2024-03-23T13:23:39.321+0000 7f05c419b700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:12.370423+0000 front 2024-03-23T13:23:12.370460+0000 (oldest deadline 2024-03-23T13:23:35.870389+0000) 2024-03-23T13:23:40.122 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:39 smithi007 podman[94281]: 2024-03-23 13:23:39.84551703 +0000 UTC m=+5.470495500 container died bf2e9ff3f6c46229f207938bfcbe4c973388bd2f878805527ee6b76347c059cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-23T13:23:40.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.998+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:40.466 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.998+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:40.466 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.998+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:40.466 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.998+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:40.467 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.998+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:40.467 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.998+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:40.467 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:39 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:39.998+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:13.679969+0000 front 2024-03-23T13:23:13.680051+0000 (oldest deadline 2024-03-23T13:23:37.779926+0000) 2024-03-23T13:23:40.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:40 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:40.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:40.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:40 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:40.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:40.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:40 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:40.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:40.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:40 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:40.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:40.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:40 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:40.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:40.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:40 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:40.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:40.547 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:40 smithi007 podman[94281]: 2024-03-23 13:23:40.317999274 +0000 UTC m=+5.942977754 container cleanup bf2e9ff3f6c46229f207938bfcbe4c973388bd2f878805527ee6b76347c059cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2) 2024-03-23T13:23:40.547 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:40 smithi007 bash[94281]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9 2024-03-23T13:23:40.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:40 smithi007 podman[94298]: 2024-03-23 13:23:40.544357711 +0000 UTC m=+0.692634321 container remove bf2e9ff3f6c46229f207938bfcbe4c973388bd2f878805527ee6b76347c059cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-23T13:23:41.113 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:40 smithi007 podman[94387]: 2024-03-23 13:23:40.761216888 +0000 UTC m=+0.017619247 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:23:41.113 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:40 smithi007 podman[94387]: 2024-03-23 13:23:40.887232065 +0000 UTC m=+0.143634465 container create fa5a0af3a30005a866d5de8d952ce14988ab4a0633fd8ecc9adb8f81c4ee02be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-23T13:23:41.113 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:41 smithi007 podman[94387]: 2024-03-23 13:23:41.106632905 +0000 UTC m=+0.363035277 container init fa5a0af3a30005a866d5de8d952ce14988ab4a0633fd8ecc9adb8f81c4ee02be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:23:41.113 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:41 smithi007 podman[94387]: 2024-03-23 13:23:41.110063357 +0000 UTC m=+0.366465715 container start fa5a0af3a30005a866d5de8d952ce14988ab4a0633fd8ecc9adb8f81c4ee02be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-23T13:23:41.414 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:41 smithi007 podman[94387]: 2024-03-23 13:23:41.206474571 +0000 UTC m=+0.462876939 container attach fa5a0af3a30005a866d5de8d952ce14988ab4a0633fd8ecc9adb8f81c4ee02be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-23T13:23:41.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:41 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:41.038+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:41.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:41 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:41.038+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:41.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:41 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:41.038+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:41.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:41 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:41.038+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:41.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:41 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:41.038+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:41.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:41 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:41.038+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:41.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:41 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:41.038+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:13.679969+0000 front 2024-03-23T13:23:13.680051+0000 (oldest deadline 2024-03-23T13:23:37.779926+0000) 2024-03-23T13:23:41.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:41 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:41.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:41.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:41 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:41.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:41.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:41 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:41.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:41.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:41 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:41.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:41.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:41 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:41.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:41.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:41 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:41.230+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:41.697 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 23 13:23:41 smithi007 podman[94387]: 2024-03-23 13:23:41.41039903 +0000 UTC m=+0.666801389 container died fa5a0af3a30005a866d5de8d952ce14988ab4a0633fd8ecc9adb8f81c4ee02be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:23:42.025 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.9.service' 2024-03-23T13:23:42.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:23:42.068 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-23T13:23:42.069 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-23T13:23:42.069 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.10 2024-03-23T13:23:42.391 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:42.065+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:42.391 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:42.065+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:42.391 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:42.065+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:42.391 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:42.065+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:42.391 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:42.065+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:42.392 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:42.065+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:42.392 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:42.065+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:13.679969+0000 front 2024-03-23T13:23:13.680051+0000 (oldest deadline 2024-03-23T13:23:37.779926+0000) 2024-03-23T13:23:42.392 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 systemd[1]: Stopping Ceph osd.10 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:23:42.392 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:42.386+0000 7fa8994ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:23:42.392 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:42.386+0000 7fa8994ad700 -1 osd.10 90 *** Got signal Terminated *** 2024-03-23T13:23:42.392 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:42 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:42.386+0000 7fa8994ad700 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:23:42.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:42 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:42.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:42.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:42 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:42.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:42.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:42 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:42.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:42.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:42 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:42.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:42.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:42 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:42.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:42.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:42 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:42.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:43.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:43 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:43.064+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:43.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:43 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:43.064+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:43.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:43 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:43.064+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:43.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:43 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:43.064+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:43.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:43 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:43.064+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:43.450 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:43 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:43.064+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:43.450 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:43 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:43.064+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:13.679969+0000 front 2024-03-23T13:23:13.680051+0000 (oldest deadline 2024-03-23T13:23:37.779926+0000) 2024-03-23T13:23:43.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:43 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:43.260+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:43.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:43 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:43.260+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:43.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:43 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:43.260+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:43.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:43 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:43.260+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:43.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:43 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:43.260+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:43.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:43 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:43.260+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:44.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:44 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:44.060+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:44.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:44 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:44.060+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:44.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:44 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:44.060+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:44.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:44 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:44.060+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:44.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:44 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:44.060+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:44.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:44 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:44.060+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:44.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:44 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:44.060+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:13.679969+0000 front 2024-03-23T13:23:13.680051+0000 (oldest deadline 2024-03-23T13:23:37.779926+0000) 2024-03-23T13:23:44.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:44 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:44.274+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:44.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:44 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:44.274+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:44.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:44 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:44.274+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:44.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:44 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:44.274+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:44.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:44 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:44.274+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:44.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:44 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:44.274+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:45.409 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:45 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:45.013+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:45.410 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:45 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:45.013+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:45.410 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:45 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:45.013+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:45.410 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:45 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:45.013+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:45.410 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:45 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:45.013+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:45.410 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:45 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:45.013+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:45.410 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:45 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:45.013+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:13.679969+0000 front 2024-03-23T13:23:13.680051+0000 (oldest deadline 2024-03-23T13:23:37.779926+0000) 2024-03-23T13:23:45.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:45 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:45.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:45.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:45 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:45.295+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:45.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:45 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:45.295+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:45.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:45 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:45.295+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:45.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:45 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:45.295+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:45.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:45 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:45.295+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:45.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:45 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:45.295+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:46.448 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:46 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:46.031+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:46.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:46 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:46.031+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:46.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:46 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:46.031+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:46.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:46 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:46.031+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:46.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:46 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:46.031+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:46.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:46 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:46.031+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:46.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:46 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:46.031+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:13.679969+0000 front 2024-03-23T13:23:13.680051+0000 (oldest deadline 2024-03-23T13:23:37.779926+0000) 2024-03-23T13:23:46.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:46 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:46.318+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:46.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:46 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:46.318+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:46.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:46 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:46.318+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:46.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:46 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:46.318+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:46.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:46 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:46.318+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:46.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:46 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:46.318+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:46.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:46 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:46.318+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:47.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:47.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:20.771669+0000 front 2024-03-23T13:22:20.771638+0000 (oldest deadline 2024-03-23T13:22:46.671598+0000) 2024-03-23T13:23:47.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:47.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:32.473047+0000 front 2024-03-23T13:22:32.473132+0000 (oldest deadline 2024-03-23T13:22:55.373208+0000) 2024-03-23T13:23:47.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:47.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:40.674262+0000 front 2024-03-23T13:22:40.674322+0000 (oldest deadline 2024-03-23T13:23:04.174232+0000) 2024-03-23T13:23:47.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:47.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-23T13:22:47.575713+0000 front 2024-03-23T13:22:47.575735+0000 (oldest deadline 2024-03-23T13:23:12.875751+0000) 2024-03-23T13:23:47.449 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:47.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:58.677013+0000 front 2024-03-23T13:22:58.676988+0000 (oldest deadline 2024-03-23T13:23:23.977077+0000) 2024-03-23T13:23:47.450 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:47.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.078961+0000 front 2024-03-23T13:23:06.078871+0000 (oldest deadline 2024-03-23T13:23:31.378845+0000) 2024-03-23T13:23:47.450 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:47 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10[61781]: 2024-03-23T13:23:47.079+0000 7fa895aa1700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:13.679969+0000 front 2024-03-23T13:23:13.680051+0000 (oldest deadline 2024-03-23T13:23:37.779926+0000) 2024-03-23T13:23:47.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:47 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:47.341+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:47.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:47 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:47.341+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:47.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:47 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:47.341+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:47.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:47 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:47.341+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:47.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:47 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:47.341+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:47.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:47 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:47.341+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:47.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:47 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:47.341+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:48.198 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:47 smithi130 podman[70942]: 2024-03-23 13:23:47.90659539 +0000 UTC m=+5.647870750 container died b5373872d659509b8921d6760c3b9f6e158e4cbee3a4486b478551a4895f5144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:23:48.698 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:48 smithi130 podman[70942]: 2024-03-23 13:23:48.22185835 +0000 UTC m=+5.963133700 container cleanup b5373872d659509b8921d6760c3b9f6e158e4cbee3a4486b478551a4895f5144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-23T13:23:48.699 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:48 smithi130 bash[70942]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10 2024-03-23T13:23:48.699 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:48 smithi130 podman[70961]: 2024-03-23 13:23:48.446074076 +0000 UTC m=+0.535160243 container remove b5373872d659509b8921d6760c3b9f6e158e4cbee3a4486b478551a4895f5144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-23T13:23:48.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:48 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:48.310+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:48.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:48 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:48.310+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:48.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:48 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:48.310+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:48.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:48 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:48.310+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:48.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:48 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:48.310+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:48.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:48 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:48.310+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:48.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:48 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:48.310+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:49.103 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:48 smithi130 podman[71049]: 2024-03-23 13:23:48.739118104 +0000 UTC m=+0.017816738 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:23:49.103 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:48 smithi130 podman[71049]: 2024-03-23 13:23:48.905135466 +0000 UTC m=+0.183834095 container create 120f5c8aa0fdb4913ad2b53396163c81c7d980807e304278dbc73e8da4a216c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:23:49.103 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:49 smithi130 podman[71049]: 2024-03-23 13:23:49.099744186 +0000 UTC m=+0.378442830 container init 120f5c8aa0fdb4913ad2b53396163c81c7d980807e304278dbc73e8da4a216c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0) 2024-03-23T13:23:49.404 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:49 smithi130 podman[71049]: 2024-03-23 13:23:49.10339251 +0000 UTC m=+0.382091132 container start 120f5c8aa0fdb4913ad2b53396163c81c7d980807e304278dbc73e8da4a216c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:23:49.404 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:49 smithi130 podman[71049]: 2024-03-23 13:23:49.182514261 +0000 UTC m=+0.461212909 container attach 120f5c8aa0fdb4913ad2b53396163c81c7d980807e304278dbc73e8da4a216c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:23:49.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:49 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:49.275+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:49.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:49 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:49.275+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:49.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:49 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:49.275+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:49.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:49 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:49.275+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:49.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:49 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:49.275+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:49.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:49 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:49.275+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:49.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:49 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:49.275+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:49.697 INFO:journalctl@ceph.osd.10.smithi130.stdout:Mar 23 13:23:49 smithi130 podman[71049]: 2024-03-23 13:23:49.400329397 +0000 UTC m=+0.679028028 container died 120f5c8aa0fdb4913ad2b53396163c81c7d980807e304278dbc73e8da4a216c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:23:49.969 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.10.service' 2024-03-23T13:23:50.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:23:50.015 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-23T13:23:50.015 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-23T13:23:50.016 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.11 2024-03-23T13:23:50.232 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 systemd[1]: Stopping Ceph osd.11 for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:23:50.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:50.228+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:50.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:50.228+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:50.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:50.228+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:50.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:50.228+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:50.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:50.228+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:50.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:50.228+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:50.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:50.228+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:50.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:50.323+0000 7f4f8e05b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:23:50.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:50.323+0000 7f4f8e05b700 -1 osd.11 90 *** Got signal Terminated *** 2024-03-23T13:23:50.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:50 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:50.323+0000 7f4f8e05b700 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T13:23:51.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:51 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:51.262+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:51.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:51 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:51.262+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:51.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:51 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:51.262+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:51.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:51 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:51.262+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:51.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:51 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:51.262+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:51.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:51 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:51.262+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:51.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:51 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:51.262+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:52.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:52.301+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:52.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:52.301+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:52.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:52.301+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:52.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:52.301+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:52.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:52.301+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:52.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:52.301+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:52.793 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:52 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:52.301+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:53.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:53.269+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:53.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:53.269+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:53.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:53.269+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:53.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:53.269+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:53.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:53.269+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:53.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:53.269+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:53.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:53 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:53.269+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:54.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:54.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:54.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:54.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:54.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:54.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:54.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:54.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:54.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:54.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:54.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:54.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:54.543 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:54 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:54.248+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:55.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:55.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-23T13:22:26.079322+0000 front 2024-03-23T13:22:26.079426+0000 (oldest deadline 2024-03-23T13:22:48.379348+0000) 2024-03-23T13:23:55.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:55.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6804 osd.1 since back 2024-03-23T13:22:31.880658+0000 front 2024-03-23T13:22:31.880378+0000 (oldest deadline 2024-03-23T13:22:55.980417+0000) 2024-03-23T13:23:55.791 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:55.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-23T13:22:37.681203+0000 front 2024-03-23T13:22:37.681282+0000 (oldest deadline 2024-03-23T13:23:05.882063+0000) 2024-03-23T13:23:55.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:55.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6812 osd.4 since back 2024-03-23T13:22:57.985481+0000 front 2024-03-23T13:22:57.985378+0000 (oldest deadline 2024-03-23T13:23:19.685453+0000) 2024-03-23T13:23:55.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:55.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-23T13:23:06.087130+0000 front 2024-03-23T13:23:06.087188+0000 (oldest deadline 2024-03-23T13:23:29.587196+0000) 2024-03-23T13:23:55.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:55.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-23T13:23:14.288144+0000 front 2024-03-23T13:23:14.288110+0000 (oldest deadline 2024-03-23T13:23:40.188132+0000) 2024-03-23T13:23:55.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:55.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.130:6820 osd.7 since back 2024-03-23T13:23:22.389755+0000 front 2024-03-23T13:23:22.389795+0000 (oldest deadline 2024-03-23T13:23:45.289713+0000) 2024-03-23T13:23:55.792 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:55 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11[61752]: 2024-03-23T13:23:55.294+0000 7f4f8a64f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.155:6820 osd.8 since back 2024-03-23T13:23:29.390604+0000 front 2024-03-23T13:23:29.390576+0000 (oldest deadline 2024-03-23T13:23:54.591216+0000) 2024-03-23T13:23:56.042 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:55 smithi155 podman[71122]: 2024-03-23 13:23:55.787873095 +0000 UTC m=+5.563371945 container died bd56daa819f562d2fa6038141527800d6b27328a201073e932146491bfe00271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319) 2024-03-23T13:23:56.378 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:56 smithi155 podman[71122]: 2024-03-23 13:23:56.182603839 +0000 UTC m=+5.958102686 container cleanup bd56daa819f562d2fa6038141527800d6b27328a201073e932146491bfe00271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:23:56.379 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:56 smithi155 bash[71122]: ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11 2024-03-23T13:23:56.762 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:56 smithi155 podman[71145]: 2024-03-23 13:23:56.375188958 +0000 UTC m=+0.585994292 container remove bd56daa819f562d2fa6038141527800d6b27328a201073e932146491bfe00271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-23T13:23:57.042 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:56 smithi155 podman[71240]: 2024-03-23 13:23:56.759185555 +0000 UTC m=+0.103311743 container create 70e92e83b3f9d962c899a4dae081075a13536dd4d3f80f1d56a698c3d6cddc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-23T13:23:57.042 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:56 smithi155 podman[71240]: 2024-03-23 13:23:56.673424333 +0000 UTC m=+0.017550536 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T13:23:57.042 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:56 smithi155 podman[71240]: 2024-03-23 13:23:56.962402858 +0000 UTC m=+0.306529053 container init 70e92e83b3f9d962c899a4dae081075a13536dd4d3f80f1d56a698c3d6cddc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:23:57.042 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:56 smithi155 podman[71240]: 2024-03-23 13:23:56.965632934 +0000 UTC m=+0.309759133 container start 70e92e83b3f9d962c899a4dae081075a13536dd4d3f80f1d56a698c3d6cddc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-23T13:23:57.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:57 smithi155 podman[71240]: 2024-03-23 13:23:57.069671571 +0000 UTC m=+0.413797772 container attach 70e92e83b3f9d962c899a4dae081075a13536dd4d3f80f1d56a698c3d6cddc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-23T13:23:57.542 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 23 13:23:57 smithi155 podman[71240]: 2024-03-23 13:23:57.251723671 +0000 UTC m=+0.595849867 container died 70e92e83b3f9d962c899a4dae081075a13536dd4d3f80f1d56a698c3d6cddc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319) 2024-03-23T13:23:57.836 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@osd.11.service' 2024-03-23T13:23:57.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:23:57.878 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-23T13:23:57.879 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-23T13:23:57.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.a 2024-03-23T13:23:58.183 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:23:57 smithi007 systemd[1]: Stopping Ceph mds.a for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:23:58.581 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:23:58 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a[73759]: 2024-03-23T13:23:58.178+0000 7f8434859700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:23:58.581 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:23:58 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a[73759]: 2024-03-23T13:23:58.178+0000 7f8434859700 -1 mds.a *** got signal Terminated *** 2024-03-23T13:24:01.549 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:24:01 smithi007 podman[94504]: 2024-03-23 13:24:01.255518609 +0000 UTC m=+3.185554321 container died 256ad5530e33c8f461e024fb0d6e7d47a4c27bbb261367f2a79b71c58a7259a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:24:01.831 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:24:01 smithi007 podman[94504]: 2024-03-23 13:24:01.736937686 +0000 UTC m=+3.666973397 container cleanup 256ad5530e33c8f461e024fb0d6e7d47a4c27bbb261367f2a79b71c58a7259a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-23T13:24:01.831 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 23 13:24:01 smithi007 bash[94504]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-a 2024-03-23T13:24:02.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.a.service' 2024-03-23T13:24:02.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:02.159 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-23T13:24:02.159 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-23T13:24:02.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.d 2024-03-23T13:24:02.581 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:24:02 smithi007 systemd[1]: Stopping Ceph mds.d for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:02.581 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:24:02 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-d[72801]: 2024-03-23T13:24:02.491+0000 7f77518d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:02.582 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:24:02 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-d[72801]: 2024-03-23T13:24:02.491+0000 7f77518d0700 -1 mds.d *** got signal Terminated *** 2024-03-23T13:24:06.621 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:24:06 smithi007 podman[94626]: 2024-03-23 13:24:06.366935467 +0000 UTC m=+3.985857609 container died d1feeb764dee2c0c5f32ae9182c6e7f672f8fc68f8acd2652b64ea404386baf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:24:06.996 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:24:06 smithi007 podman[94626]: 2024-03-23 13:24:06.808164889 +0000 UTC m=+4.427087076 container cleanup d1feeb764dee2c0c5f32ae9182c6e7f672f8fc68f8acd2652b64ea404386baf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T13:24:06.996 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:24:06 smithi007 bash[94626]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-d 2024-03-23T13:24:07.247 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:24:06 smithi007 podman[94640]: 2024-03-23 13:24:06.992879219 +0000 UTC m=+0.620834667 container remove d1feeb764dee2c0c5f32ae9182c6e7f672f8fc68f8acd2652b64ea404386baf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-23T13:24:07.247 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:24:07 smithi007 systemd[1]: ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.d.service: Deactivated successfully. 2024-03-23T13:24:07.247 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 23 13:24:07 smithi007 systemd[1]: Stopped Ceph mds.d for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:24:07.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.d.service' 2024-03-23T13:24:07.283 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:07.283 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-23T13:24:07.283 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-23T13:24:07.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.g 2024-03-23T13:24:07.581 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:24:07 smithi007 systemd[1]: Stopping Ceph mds.g for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:08.081 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:24:07 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g[73410]: 2024-03-23T13:24:07.602+0000 7fd156a23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:08.081 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:24:07 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g[73410]: 2024-03-23T13:24:07.602+0000 7fd156a23700 -1 mds.g *** got signal Terminated *** 2024-03-23T13:24:09.533 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:24:09 smithi007 podman[94747]: 2024-03-23 13:24:09.250627042 +0000 UTC m=+1.745833575 container died b0f5d7404803797bea7a4ff01baab27a6119f32ab6007eec0724650fe2670fc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-03-23T13:24:09.831 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:24:09 smithi007 podman[94747]: 2024-03-23 13:24:09.628890252 +0000 UTC m=+2.124096785 container cleanup b0f5d7404803797bea7a4ff01baab27a6119f32ab6007eec0724650fe2670fc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-23T13:24:09.831 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 23 13:24:09 smithi007 bash[94747]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-g 2024-03-23T13:24:10.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.g.service' 2024-03-23T13:24:10.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:10.087 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-23T13:24:10.087 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-23T13:24:10.087 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.j 2024-03-23T13:24:10.444 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:24:10 smithi007 systemd[1]: Stopping Ceph mds.j for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:10.831 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:24:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-j[72449]: 2024-03-23T13:24:10.440+0000 7f22cfad5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:10.831 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:24:10 smithi007 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-j[72449]: 2024-03-23T13:24:10.440+0000 7f22cfad5700 -1 mds.j *** got signal Terminated *** 2024-03-23T13:24:11.695 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:24:11 smithi007 podman[94866]: 2024-03-23 13:24:11.436033036 +0000 UTC m=+1.135163106 container died 287fc6b1b0f54ddda2fb3d0cd5a60de422d1b188198f5de8a9e557c9a0895698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-23T13:24:12.063 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:24:11 smithi007 podman[94866]: 2024-03-23 13:24:11.866775235 +0000 UTC m=+1.565905313 container cleanup 287fc6b1b0f54ddda2fb3d0cd5a60de422d1b188198f5de8a9e557c9a0895698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-j, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-23T13:24:12.063 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 23 13:24:11 smithi007 bash[94866]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-j 2024-03-23T13:24:12.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.j.service' 2024-03-23T13:24:12.302 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:12.302 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-23T13:24:12.302 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-23T13:24:12.303 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.b 2024-03-23T13:24:12.598 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:24:12 smithi130 systemd[1]: Stopping Ceph mds.b for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:12.857 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:24:12 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: 2024-03-23T13:24:12.594+0000 7f73fb147700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:12.857 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:24:12 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b[67033]: 2024-03-23T13:24:12.594+0000 7f73fb147700 -1 mds.b *** got signal Terminated *** 2024-03-23T13:24:16.949 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:24:16 smithi130 podman[71154]: 2024-03-23 13:24:16.604591074 +0000 UTC m=+4.117265164 container died 4fde6ee0de3a642b8ca2f9b83225217aec277c24529e913b61689b980196ecd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-23T13:24:17.357 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:24:17 smithi130 podman[71154]: 2024-03-23 13:24:17.135267228 +0000 UTC m=+4.647941310 container cleanup 4fde6ee0de3a642b8ca2f9b83225217aec277c24529e913b61689b980196ecd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-23T13:24:17.357 INFO:journalctl@ceph.mds.b.smithi130.stdout:Mar 23 13:24:17 smithi130 bash[71154]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-b 2024-03-23T13:24:17.565 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.b.service' 2024-03-23T13:24:17.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:17.603 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-23T13:24:17.603 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-23T13:24:17.603 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.e 2024-03-23T13:24:17.926 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:24:17 smithi130 systemd[1]: Stopping Ceph mds.e for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:18.199 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:24:17 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e[66405]: 2024-03-23T13:24:17.921+0000 7f5777e49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:18.199 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:24:17 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e[66405]: 2024-03-23T13:24:17.921+0000 7f5777e49700 -1 mds.e *** got signal Terminated *** 2024-03-23T13:24:21.543 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:24:21 smithi130 podman[71287]: 2024-03-23 13:24:21.265016946 +0000 UTC m=+3.451058982 container died c0da684a212996c9e2fae0ce5a73a08f0bddb0cb0859815982e5cfe5ad54e6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-23T13:24:21.851 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:24:21 smithi130 podman[71287]: 2024-03-23 13:24:21.630206357 +0000 UTC m=+3.816248390 container cleanup c0da684a212996c9e2fae0ce5a73a08f0bddb0cb0859815982e5cfe5ad54e6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-23T13:24:21.851 INFO:journalctl@ceph.mds.e.smithi130.stdout:Mar 23 13:24:21 smithi130 bash[71287]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-e 2024-03-23T13:24:21.978 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.e.service' 2024-03-23T13:24:22.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:22.013 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-23T13:24:22.013 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-23T13:24:22.013 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.h 2024-03-23T13:24:22.449 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:24:22 smithi130 systemd[1]: Stopping Ceph mds.h for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:22.449 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:24:22 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h[66714]: 2024-03-23T13:24:22.316+0000 7ff57a265700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:22.450 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:24:22 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h[66714]: 2024-03-23T13:24:22.316+0000 7ff57a265700 -1 mds.h *** got signal Terminated *** 2024-03-23T13:24:25.521 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:24:25 smithi130 podman[71409]: 2024-03-23 13:24:25.267476598 +0000 UTC m=+3.033322568 container died 189dca2e132be202fd169aec61d53072db4a3438cfc77551ba86586d823e672b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-23T13:24:25.780 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:24:25 smithi130 podman[71409]: 2024-03-23 13:24:25.617198781 +0000 UTC m=+3.383044758 container cleanup 189dca2e132be202fd169aec61d53072db4a3438cfc77551ba86586d823e672b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-23T13:24:25.780 INFO:journalctl@ceph.mds.h.smithi130.stdout:Mar 23 13:24:25 smithi130 bash[71409]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-h 2024-03-23T13:24:25.999 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.h.service' 2024-03-23T13:24:26.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:26.033 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-23T13:24:26.033 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-23T13:24:26.033 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.k 2024-03-23T13:24:26.349 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:24:26 smithi130 systemd[1]: Stopping Ceph mds.k for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:26.699 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:24:26 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k[66083]: 2024-03-23T13:24:26.344+0000 7fc89ae55700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:26.699 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:24:26 smithi130 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k[66083]: 2024-03-23T13:24:26.344+0000 7fc89ae55700 -1 mds.k *** got signal Terminated *** 2024-03-23T13:24:29.524 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:24:29 smithi130 podman[71531]: 2024-03-23 13:24:29.267049139 +0000 UTC m=+3.020587842 container died 0c546a506d7eff7d991eaaa0036f46a923d761ab2f47a4badb40cb04bac60c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-23T13:24:29.799 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:24:29 smithi130 podman[71531]: 2024-03-23 13:24:29.611782813 +0000 UTC m=+3.365321507 container cleanup 0c546a506d7eff7d991eaaa0036f46a923d761ab2f47a4badb40cb04bac60c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-23T13:24:29.800 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:24:29 smithi130 bash[71531]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k 2024-03-23T13:24:30.035 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.k.service' 2024-03-23T13:24:30.058 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:24:29 smithi130 podman[71545]: 2024-03-23 13:24:29.796069096 +0000 UTC m=+0.528141924 container remove 0c546a506d7eff7d991eaaa0036f46a923d761ab2f47a4badb40cb04bac60c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-k, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:24:30.058 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:24:30 smithi130 systemd[1]: ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.k.service: Deactivated successfully. 2024-03-23T13:24:30.058 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:24:30 smithi130 systemd[1]: Stopped Ceph mds.k for 745f591a-e916-11ee-95cd-87774f69a715. 2024-03-23T13:24:30.059 INFO:journalctl@ceph.mds.k.smithi130.stdout:Mar 23 13:24:30 smithi130 systemd[1]: ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.k.service: Consumed 1.704s CPU time. 2024-03-23T13:24:30.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:30.069 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-23T13:24:30.069 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-23T13:24:30.069 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.c 2024-03-23T13:24:30.361 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:24:30 smithi155 systemd[1]: Stopping Ceph mds.c for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:30.792 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:24:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c[66850]: 2024-03-23T13:24:30.356+0000 7f192244f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:30.793 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:24:30 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c[66850]: 2024-03-23T13:24:30.356+0000 7f192244f700 -1 mds.c *** got signal Terminated *** 2024-03-23T13:24:33.527 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:24:33 smithi155 podman[71345]: 2024-03-23 13:24:33.267032528 +0000 UTC m=+3.005639514 container died 17af4a18e4c41fc31d68f58bb824873d913e542516da0df6ee58e196dd4f3a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-23T13:24:33.792 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:24:33 smithi155 podman[71345]: 2024-03-23 13:24:33.614976488 +0000 UTC m=+3.353583461 container cleanup 17af4a18e4c41fc31d68f58bb824873d913e542516da0df6ee58e196dd4f3a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-23T13:24:33.793 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 23 13:24:33 smithi155 bash[71345]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-c 2024-03-23T13:24:34.047 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.c.service' 2024-03-23T13:24:34.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:34.086 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-23T13:24:34.086 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-23T13:24:34.087 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.f 2024-03-23T13:24:34.406 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:24:34 smithi155 systemd[1]: Stopping Ceph mds.f for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:34.792 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:24:34 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f[66540]: 2024-03-23T13:24:34.402+0000 7fc003244700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:34.793 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:24:34 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f[66540]: 2024-03-23T13:24:34.402+0000 7fc003244700 -1 mds.f *** got signal Terminated *** 2024-03-23T13:24:37.542 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:24:37 smithi155 podman[71480]: 2024-03-23 13:24:37.263259224 +0000 UTC m=+2.960253550 container died 952c73e93ab4205643e2545c73e441fb31b8c07bbb3bd5989c26edf24553b439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T13:24:37.892 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:24:37 smithi155 podman[71480]: 2024-03-23 13:24:37.620575154 +0000 UTC m=+3.317569453 container cleanup 952c73e93ab4205643e2545c73e441fb31b8c07bbb3bd5989c26edf24553b439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-23T13:24:37.892 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 23 13:24:37 smithi155 bash[71480]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-f 2024-03-23T13:24:38.042 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.f.service' 2024-03-23T13:24:38.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:38.078 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-23T13:24:38.078 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-23T13:24:38.078 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.i 2024-03-23T13:24:38.542 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:24:38 smithi155 systemd[1]: Stopping Ceph mds.i for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:38.543 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:24:38 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i[67167]: 2024-03-23T13:24:38.381+0000 7f4bb0774700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:38.543 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:24:38 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i[67167]: 2024-03-23T13:24:38.381+0000 7f4bb0774700 -1 mds.i *** got signal Terminated *** 2024-03-23T13:24:41.536 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:24:41 smithi155 podman[71600]: 2024-03-23 13:24:41.261157363 +0000 UTC m=+2.965088626 container died 255fec02aba7fb1e767d97f158eab740fcb4654f4c22802d00235a58ebbb90d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T13:24:41.792 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:24:41 smithi155 podman[71600]: 2024-03-23 13:24:41.648571507 +0000 UTC m=+3.352502770 container cleanup 255fec02aba7fb1e767d97f158eab740fcb4654f4c22802d00235a58ebbb90d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T13:24:41.793 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 23 13:24:41 smithi155 bash[71600]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-i 2024-03-23T13:24:42.064 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.i.service' 2024-03-23T13:24:42.099 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:42.099 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-23T13:24:42.099 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-23T13:24:42.100 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.l 2024-03-23T13:24:42.543 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:24:42 smithi155 systemd[1]: Stopping Ceph mds.l for 745f591a-e916-11ee-95cd-87774f69a715... 2024-03-23T13:24:42.543 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:24:42 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l[66225]: 2024-03-23T13:24:42.401+0000 7f6a23692700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T13:24:42.543 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:24:42 smithi155 ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l[66225]: 2024-03-23T13:24:42.401+0000 7f6a23692700 -1 mds.l *** got signal Terminated *** 2024-03-23T13:24:45.531 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:24:45 smithi155 podman[71721]: 2024-03-23 13:24:45.263006298 +0000 UTC m=+2.951321202 container died c3a25fe4bd6d9f8681c5ae94057969eff11f3e7dfcdd4c12b22aa3222e4d124a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-23T13:24:45.793 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:24:45 smithi155 podman[71721]: 2024-03-23 13:24:45.627129785 +0000 UTC m=+3.315444680 container cleanup c3a25fe4bd6d9f8681c5ae94057969eff11f3e7dfcdd4c12b22aa3222e4d124a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-23T13:24:45.793 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 23 13:24:45 smithi155 bash[71721]: ceph-745f591a-e916-11ee-95cd-87774f69a715-mds-l 2024-03-23T13:24:46.003 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-745f591a-e916-11ee-95cd-87774f69a715@mds.l.service' 2024-03-23T13:24:46.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T13:24:46.036 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-23T13:24:46.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 745f591a-e916-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T13:24:46.233 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:24:48.793 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 745f591a-e916-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T13:24:48.984 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:24:51.508 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 745f591a-e916-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T13:24:51.697 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:24:54.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T13:24:54.173 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T13:24:54.207 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T13:24:54.236 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-23T13:24:54.238 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291/remote/smithi007/crash 2024-03-23T13:24:54.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/crash -- . 2024-03-23T13:24:54.277 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T13:24:54.277 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now 2024-03-23T13:24:54.279 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291/remote/smithi130/crash 2024-03-23T13:24:54.280 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/crash -- . 2024-03-23T13:24:54.397 INFO:teuthology.orchestra.run.smithi130.stderr:tar: /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T13:24:54.398 INFO:teuthology.orchestra.run.smithi130.stderr:tar: Error is not recoverable: exiting now 2024-03-23T13:24:54.399 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291/remote/smithi155/crash 2024-03-23T13:24:54.400 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/crash -- . 2024-03-23T13:24:54.433 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/745f591a-e916-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T13:24:54.433 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-03-23T13:24:54.434 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-23T13:24:54.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-23T13:24:54.477 INFO:tasks.cephadm:Compressing logs... 2024-03-23T13:24:54.477 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T13:24:54.519 DEBUG:teuthology.orchestra.run.smithi130:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T13:24:54.522 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T13:24:54.546 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-23T13:24:54.546 INFO:teuthology.orchestra.run.smithi007.stderr:: No such file or directory 2024-03-23T13:24:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-23T13:24:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-23T13:24:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.6.log: /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-23T13:24:54.549 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.g.log 2024-03-23T13:24:54.549 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.j.log 2024-03-23T13:24:54.549 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.audit.log: /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-23T13:24:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-23T13:24:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-23T13:24:54.562 INFO:teuthology.orchestra.run.smithi130.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-23T13:24:54.562 INFO:teuthology.orchestra.run.smithi130.stderr:: No such file or directory 2024-03-23T13:24:54.562 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.10.log 2024-03-23T13:24:54.562 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-23T13:24:54.562 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.b.log 2024-03-23T13:24:54.562 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-23T13:24:54.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-23T13:24:54.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.b.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-23T13:24:54.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.k.log 2024-03-23T13:24:54.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.h.log 2024-03-23T13:24:54.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-23T13:24:54.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.e.log 2024-03-23T13:24:54.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.7.log: 90.1% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-23T13:24:54.563 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-23T13:24:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mgr.x.log: 90.0% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-23T13:24:54.565 INFO:teuthology.orchestra.run.smithi007.stderr: 94.6% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.j.log.gz 2024-03-23T13:24:54.565 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-23T13:24:54.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.log 2024-03-23T13:24:54.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.cephadm.log: 82.9% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-23T13:24:54.565 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-23T13:24:54.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-23T13:24:54.565 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.9.log 2024-03-23T13:24:54.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-volume.log: 85.9% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-23T13:24:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.d.log 2024-03-23T13:24:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-23T13:24:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:: No such file or directory 2024-03-23T13:24:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.l.log 2024-03-23T13:24:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-23T13:24:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:gzip/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.l.log: -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.i.log 2024-03-23T13:24:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.11.log 2024-03-23T13:24:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mgr.z.log 2024-03-23T13:24:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.i.log: /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.11.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-23T13:24:54.570 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.8.log 2024-03-23T13:24:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-23T13:24:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.f.log 2024-03-23T13:24:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-23T13:24:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.f.log: 90.1% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-23T13:24:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.log 2024-03-23T13:24:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.cephadm.log: 82.8% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-23T13:24:54.571 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-23T13:24:54.572 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.log: 92.2% 85.8% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mgr.z.log.gz 2024-03-23T13:24:54.572 INFO:teuthology.orchestra.run.smithi155.stderr: -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-23T13:24:54.572 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.c.log 2024-03-23T13:24:54.572 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-23T13:24:54.572 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-23T13:24:54.573 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-23T13:24:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.a.log 2024-03-23T13:24:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-23T13:24:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.d.log.gz 2024-03-23T13:24:54.581 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mgr.y.log: 92.0%gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.log 2024-03-23T13:24:54.581 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-23T13:24:54.588 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.cephadm.log: 82.8%gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-23T13:24:54.594 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.log: -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-23T13:24:54.595 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-23T13:24:54.600 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-23T13:24:54.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-volume.log: 85.8% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-23T13:24:54.628 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-23T13:24:54.749 INFO:teuthology.orchestra.run.smithi007.stderr: 89.9% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-23T13:24:54.751 INFO:teuthology.orchestra.run.smithi007.stderr: 94.5% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.g.log.gz 2024-03-23T13:24:54.764 INFO:teuthology.orchestra.run.smithi155.stderr: 94.7% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.l.log.gz 2024-03-23T13:24:54.792 INFO:teuthology.orchestra.run.smithi007.stderr: 94.4% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.a.log.gz 2024-03-23T13:24:54.892 INFO:teuthology.orchestra.run.smithi130.stderr: 94.6% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.k.log.gz 2024-03-23T13:24:54.935 INFO:teuthology.orchestra.run.smithi155.stderr: 91.9% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-23T13:24:54.945 INFO:teuthology.orchestra.run.smithi130.stderr: 91.9% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-23T13:24:55.108 INFO:teuthology.orchestra.run.smithi007.stderr: 90.6% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-23T13:24:55.747 INFO:teuthology.orchestra.run.smithi130.stderr: 95.0% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.e.log.gz 2024-03-23T13:24:56.085 INFO:teuthology.orchestra.run.smithi007.stderr: 92.8% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.9.log.gz 2024-03-23T13:24:56.086 INFO:teuthology.orchestra.run.smithi155.stderr: 94.1% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.i.log.gz 2024-03-23T13:24:56.114 INFO:teuthology.orchestra.run.smithi130.stderr: 92.3% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-23T13:24:56.144 INFO:teuthology.orchestra.run.smithi130.stderr: 94.3% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.h.log.gz 2024-03-23T13:24:56.239 INFO:teuthology.orchestra.run.smithi007.stderr: 92.0% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-23T13:24:56.277 INFO:teuthology.orchestra.run.smithi155.stderr: 92.1% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.11.log.gz 2024-03-23T13:24:56.376 INFO:teuthology.orchestra.run.smithi130.stderr: 92.2% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.10.log.gz 2024-03-23T13:24:56.389 INFO:teuthology.orchestra.run.smithi155.stderr: 94.0% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.f.log.gz 2024-03-23T13:24:56.464 INFO:teuthology.orchestra.run.smithi155.stderr: 94.2% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.c.log.gz 2024-03-23T13:24:56.489 INFO:teuthology.orchestra.run.smithi155.stderr: 92.1% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.8.log.gz 2024-03-23T13:24:56.503 INFO:teuthology.orchestra.run.smithi007.stderr: 92.5% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-23T13:24:56.560 INFO:teuthology.orchestra.run.smithi155.stderr: 92.0% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-23T13:24:56.614 INFO:teuthology.orchestra.run.smithi007.stderr: 92.4% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-23T13:24:56.616 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-03-23T13:24:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m2.084s 2024-03-23T13:24:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m8.194s 2024-03-23T13:24:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.284s 2024-03-23T13:24:56.891 INFO:teuthology.orchestra.run.smithi130.stderr: 92.6% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-23T13:24:56.899 INFO:teuthology.orchestra.run.smithi130.stderr: 92.5% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-23T13:24:56.927 INFO:teuthology.orchestra.run.smithi155.stderr: 92.7% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-23T13:24:56.929 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-03-23T13:24:56.929 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m2.393s 2024-03-23T13:24:56.929 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m13.327s 2024-03-23T13:24:56.929 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m0.406s 2024-03-23T13:25:04.678 INFO:teuthology.orchestra.run.smithi130.stderr: 93.6% -- replaced with /var/log/ceph/745f591a-e916-11ee-95cd-87774f69a715/ceph-mds.b.log.gz 2024-03-23T13:25:04.680 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-03-23T13:25:04.680 INFO:teuthology.orchestra.run.smithi130.stderr:real 0m10.144s 2024-03-23T13:25:04.680 INFO:teuthology.orchestra.run.smithi130.stderr:user 0m20.705s 2024-03-23T13:25:04.680 INFO:teuthology.orchestra.run.smithi130.stderr:sys 0m0.662s 2024-03-23T13:25:04.681 INFO:tasks.cephadm:Archiving logs... 2024-03-23T13:25:04.681 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291/remote/smithi007/log 2024-03-23T13:25:04.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T13:25:05.885 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291/remote/smithi130/log 2024-03-23T13:25:05.886 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T13:25:08.633 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291/remote/smithi155/log 2024-03-23T13:25:08.634 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T13:25:10.174 INFO:tasks.cephadm:Removing cluster... 2024-03-23T13:25:10.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 745f591a-e916-11ee-95cd-87774f69a715 --force 2024-03-23T13:25:10.374 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:25:10.638 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 745f591a-e916-11ee-95cd-87774f69a715 --force 2024-03-23T13:25:10.828 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:25:11.100 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 745f591a-e916-11ee-95cd-87774f69a715 --force 2024-03-23T13:25:11.286 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 745f591a-e916-11ee-95cd-87774f69a715 2024-03-23T13:25:11.553 INFO:tasks.cephadm:Removing cephadm ... 2024-03-23T13:25:11.554 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T13:25:11.571 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T13:25:11.587 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T13:25:11.602 INFO:tasks.cephadm:Teardown complete 2024-03-23T13:25:11.603 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-23T13:25:11.613 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-23T13:25:11.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T13:25:11.616 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T13:25:11.629 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T13:25:11.725 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-23T13:25:11.735 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-23T13:25:11.735 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T13:25:11.738 DEBUG:teuthology.orchestra.run.smithi130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T13:25:11.740 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T13:25:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-03-23T13:25:11.752 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-03-23T13:25:11.754 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-03-23T13:25:11.755 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T13:25:11.755 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-03-23T13:25:11.756 INFO:teuthology.orchestra.run.smithi007.stdout:^- hv01.front.sepia.ceph.com 2 6 377 0 +458us[ +458us] +/- 190ms 2024-03-23T13:25:11.756 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 63 -303us[ -311us] +/- 65ms 2024-03-23T13:25:11.756 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +353us[ +345us] +/- 75ms 2024-03-23T13:25:11.756 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:25:11.757 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T13:25:11.758 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-03-23T13:25:11.758 INFO:teuthology.orchestra.run.smithi130.stdout:^- hv01.front.sepia.ceph.com 2 6 377 0 +467us[ +467us] +/- 190ms 2024-03-23T13:25:11.758 INFO:teuthology.orchestra.run.smithi130.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 -310us[ -325us] +/- 66ms 2024-03-23T13:25:11.758 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +357us[ +343us] +/- 76ms 2024-03-23T13:25:11.758 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:25:11.759 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T13:25:11.759 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-03-23T13:25:11.759 INFO:teuthology.orchestra.run.smithi155.stdout:^- hv01.front.sepia.ceph.com 2 6 377 64 +468us[ +449us] +/- 189ms 2024-03-23T13:25:11.760 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -315us[ -342us] +/- 66ms 2024-03-23T13:25:11.760 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +352us[ +352us] +/- 76ms 2024-03-23T13:25:11.760 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T13:25:11.760 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-23T13:25:11.783 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-23T13:25:11.783 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-23T13:25:11.826 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T13:25:11.911 DEBUG:teuthology.orchestra.run.smithi130:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T13:25:11.982 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T13:25:12.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T13:25:12.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:25:12.086 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T13:25:12.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:25:12.115 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T13:25:12.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:25:12.146 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-23T13:25:12.156 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-23T13:25:12.166 INFO:teuthology.task.internal:Duration was 1374.072375 seconds 2024-03-23T13:25:12.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-23T13:25:12.212 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-23T13:25:12.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T13:25:12.215 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T13:25:12.218 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T13:25:12.258 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T13:25:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T13:25:12.260 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T13:25:12.730 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-23T13:25:12.730 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-03-23T13:25:12.731 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T13:25:12.754 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi130.front.sepia.ceph.com 2024-03-23T13:25:12.755 DEBUG:teuthology.orchestra.run.smithi130:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T13:25:12.806 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-03-23T13:25:12.806 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T13:25:12.839 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-23T13:25:12.839 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T13:25:12.842 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T13:25:12.849 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T13:25:12.910 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-23T13:25:12.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T13:25:12.914 DEBUG:teuthology.orchestra.run.smithi130:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T13:25:12.918 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T13:25:13.302 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-23T13:25:13.314 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-23T13:25:13.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T13:25:13.340 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T13:25:13.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T13:25:13.394 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-23T13:25:13.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T13:25:13.410 DEBUG:teuthology.orchestra.run.smithi130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T13:25:13.412 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T13:25:13.432 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-03-23T13:25:13.434 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = core 2024-03-23T13:25:13.460 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-03-23T13:25:13.503 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T13:25:13.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:25:13.517 DEBUG:teuthology.orchestra.run.smithi130:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T13:25:13.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:25:13.531 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T13:25:13.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T13:25:13.557 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-23T13:25:13.568 INFO:teuthology.task.internal:Transferring archived files... 2024-03-23T13:25:13.569 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291/remote/smithi007 2024-03-23T13:25:13.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T13:25:13.616 DEBUG:teuthology.misc:Transferring archived files from smithi130:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291/remote/smithi130 2024-03-23T13:25:13.616 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T13:25:13.654 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291/remote/smithi155 2024-03-23T13:25:13.655 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T13:25:13.692 INFO:teuthology.task.internal:Removing archive directory... 2024-03-23T13:25:13.692 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T13:25:13.695 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T13:25:13.697 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T13:25:13.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-23T13:25:13.754 INFO:teuthology.task.internal:Not uploading archives. 2024-03-23T13:25:13.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-23T13:25:13.769 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-23T13:25:13.770 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T13:25:13.772 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T13:25:13.774 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T13:25:13.783 INFO:teuthology.orchestra.run.smithi007.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 13:25 /home/ubuntu/cephtest 2024-03-23T13:25:13.786 INFO:teuthology.orchestra.run.smithi130.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 13:25 /home/ubuntu/cephtest 2024-03-23T13:25:13.794 INFO:teuthology.orchestra.run.smithi155.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 13:25 /home/ubuntu/cephtest 2024-03-23T13:25:13.796 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-23T13:25:13.807 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-23T13:25:13.870 INFO:teuthology.nuke:Checking targets against current locks 2024-03-23T13:25:13.896 DEBUG:teuthology.nuke:shortname: smithi007 2024-03-23T13:25:13.896 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T13:25:13.920 DEBUG:teuthology.nuke:shortname: smithi130 2024-03-23T13:25:13.920 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T13:25:13.925 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 12:52:45.252911', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPPS4kGbgZtyiA+PbFpN9jAvHO0clAoZR9dS4rFiGkjqcEPhEXfJ62lARiOgP+9sWl5LUv1ttet2I5YWUeaw2MI='} 2024-03-23T13:25:13.943 DEBUG:teuthology.nuke:shortname: smithi155 2024-03-23T13:25:13.943 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T13:25:13.950 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 12:52:45.249833', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIf1WrqU4CbY9z2YzSoXc8L4M0vncfJQIXiNx6UniD6JWDZsBmBtXNdL0EaJM1pHAbuvfFyzvGsdrPDTUR0bQrM='} 2024-03-23T13:25:13.963 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618291', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 12:52:45.251610', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKdQLOmBh37R1q5FxDRS0DC7QXWyZqMOUZX08w3fbQUA2sdfLy2ujTeb2MpJqnnoZy6dpZULJ8Xc697TLbOY5e8='} 2024-03-23T13:25:13.977 INFO:teuthology.orchestra.console.smithi007:Power off 2024-03-23T13:25:13.978 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T13:25:13.998 INFO:teuthology.orchestra.console.smithi130:Power off 2024-03-23T13:25:13.999 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T13:25:14.010 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-03-23T13:25:14.011 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T13:25:14.027 DEBUG:teuthology.orchestra.console.smithi130:power off output: Chassis Power Control: Down/Off 2024-03-23T13:25:14.028 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T13:25:14.044 INFO:teuthology.orchestra.console.smithi155:Power off 2024-03-23T13:25:14.044 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T13:25:14.053 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-03-23T13:25:14.053 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-03-23T13:25:14.067 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2024-03-23T13:25:14.067 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T13:25:14.091 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-03-23T13:25:18.055 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T13:25:18.063 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T13:25:18.092 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T13:25:18.165 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-03-23T13:25:18.173 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-03-23T13:25:18.201 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-03-23T13:25:22.166 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T13:25:22.175 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T13:25:22.202 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T13:25:22.277 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-03-23T13:25:22.285 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is off 2024-03-23T13:25:22.312 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2024-03-23T13:25:22.379 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-03-23T13:25:22.386 INFO:teuthology.orchestra.console.smithi130:Power off completed 2024-03-23T13:25:22.413 INFO:teuthology.orchestra.console.smithi155:Power off completed 2024-03-23T13:25:22.519 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} duration: 1374.0723752975464 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi007 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=bdd31bdb3e27436abefdad26345b3dd4 status: fail success: false 2024-03-23T13:25:22.519 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T13:25:22.615 INFO:teuthology.run:FAIL